Cleveland Clinic Using Augmented Reality to Enhance Liver Cancer Therapy

Augmented reality is the use of computer technology to visually expand a user’s environment. It’s transforming countless industries, from gaming to retail, and, increasingly, the practice of medicine, too.

A form of augmented reality is being adapted by Cleveland Clinic research scientists and physicians for use in a wide array of clinical applications. Doctors are using a fully self-contained holographic computer –the Microsoft HoloLens augmented reality device –to enhance the treatment of liver cancer, aortic aneurysms, ovarian cancer and even face transplant surgery.

Most recently, HoloLens has been incorporated into liver cancer treatment. A Cleveland Clinic research team recently launched a first-of-its-kind clinical trial evaluating the technology for improving the accuracy and effectiveness of a minimally-invasive thermal therapy (microwave ablation) to destroy liver tumors.

“This is the first time in the world, to our knowledge, extended reality surgical navigation has been evaluated in a liver tumor ablation procedure,” said Karl West, director of medical device solutions at Cleveland Clinic’s Lerner Research Institute and one of the inventors of the technology. “HoloLens provides an entirely different way to visualize the tumor’s location. It is a paradigm shift and may have potential applications for a variety of other procedures and diseases.”

The study is comparing 2D standard of care images, displayed on a conventional flat panel screen, with 3D holographic images, which layer virtual projections on top of the user’s real-world view. This enables the medical team to see the patient’s specific anatomy in 3D. This 3D holographic image is projected onto the patient, giving the doctor “X-ray vision” to better visualize the path to the tumor.

Using the holographic image, the radiologist guides a biopsy needle into the tumor with enhanced precision. The team then delivers microwave energy through the needle, to heat and destroy the abnormal tumor tissue. Throughout the procedure, the physician uses head–mounted glasses to see the patient’s internal anatomical structures and verify tumor location with the technology’s mini-global positioning (GPS)-like system.

“We are assessing if this augmented reality system can overcome the limitations of displaying images such as ultrasound, fluoroscopy and CT on 2D flat screens,” said Charles Martin III, M.D., director of interventional oncology, in the Department of Radiology, at Cleveland Clinic. “The transformative technology allows a physician to see the patient’s anatomy in a completely unique way. I feel it can assist in enhancing minimally invasive therapies for patients. It provides an unprecedented ability to better visualize the patient’s anatomical structures and map out the most precise path to the tumor in real time.”

Advancements in Face Transplant Surgery

Cleveland Clinic’s surgical team was the first-ever to use HoloLens technology during an extremely complex total face transplant surgery, on the youngest patient in the United States to undergo the procedure.

RELATED: How Augmented Reality Shaped Cleveland Clinic’s Total Face Transplant

In 2017, the surgical team used the HoloLens headset to overlay a 3D holographic representation of the donor’s anatomy onto the recipient’s face to assess alignment and refine their surgical plans.

Accuracy in Aortic Aneurysm Treatment

In addition to liver tumor ablation, Cleveland Clinic is incorporating the virtual reality technology for aortic aneurysm treatment. Researchers are studying the ability of HoloLens to enhance accuracy in improving stent placement. Cleveland Clinic and Centerline Biomedical, a Cleveland Clinic spin-off company, were awarded a National Institutes of Health (NIH) grant to test a “holographic GPS system for the OR” that enables 3D-visualization during minimally invasive procedures.

RELATED: “GPS System for the Operating Room” to Be Tested in Patient-Specific Models

The technique includes the use of Centerline’s novel Intra-operative Positioning System (IOPS) coupled with HoloLens to visualize the procedure in true 3D (as opposed to traditional 2D X-ray). The researchers are using IOPS to control and more precisely place the stent graft within an aortic aneurysm, or other vascular lesion, thereby reducing radiation exposure and decreasing procedure time.