The phrase “It’s not brain surgery” is often used to say that a task is easy. This recognizes that neurosurgery is one of the most difficult medical fields, with potential life and death consequences.

For many years, neurosurgeons in the West have been using surgical navigation systems to perform such complex tasks and reduce the risks to patients. Despite the obvious benefits, the method was not so common in China due to infrastructure and cost issues.

The core technologies and components must be imported, at prohibitive costs, putting it out of reach for the country’s many local hospitals. The Imported surgical navigation equipment has a long learning curve and is cumbersome to be manipulated, making it even less accessible.

Developing a Low-Cost, High Accuracy Surgical Navigation System

Earlier this year, the country found a solution to the complex healthcare environment. Peking Union Medical College Hospital and Tencent AI Lab have jointly released a surgical navigation system with full intellectual property rights, consisting of a navigation tablet computer, a navigation bar, a depth-sensing camera, and other equipment, which can be mounted on a surgical head frame.

The development of a domestically produced system took three years to achieve, and costs just one-tenth of an imported version, while maintaining high accuracy.

“It is precise, portable and affordable. The system provides a real-time digital map of the brain that guides clinicians during complex surgery, significantly improves surgical reliability,” said Renzhi Wang, professor and director of the Department of Neurosurgery and doctoral advisor of Peking Union Medical College Hospital.

To date, more than 30 clinical trials have been conducted involving cerebral hemorrhage, pituitary adenoma, glioma, meningioma, craniopharyngioma and other brain-related tumors. During such operations, doctors must accurately locate lesions, while being extremely careful to avoid damage to peripheral blood vessels, nerves and other important tissues.

“In the past, we relied on our experience to create a 3D image in our mind based on a patient’s 2D CT or MRI images. This requires years of accumulated experience to do. Surgical navigation systems alleviate some of that pressure,” said Wang.

AI-powered 3D Brain Maps

Central to the success of the surgical navigation system is the use of AI and big data. Prior to surgery, the system can automatically visualize 3D MRI, CT and other scans using AI and imaging technology. A customized 3D see-through brain can be digitally generated with one button click, allowing doctors to make surgical plans and explain them to patients and their families.

The 3D see-through brain image can help doctors better analyze their patients.

During surgery, a depth-sensing camera that integrates deep learning and vision algorithms allows real-time navigation by matching the 3D brain with the patient’s actual condition, acting like a map to guide doctors in real time.

The “navigation” demo during surgery.

Image registration accuracy of the system can reach the sub-millimeter level. What’s more, improved algorithm capabilities cut the time for image registration from 30 minutes to just a few seconds. Once the image registration is complete, doctors can validate it using a navigation bar, with safeguards in place to avoid surgery occurring if there is a mismatch.

GPS System for Brain Surgery

Finally, simultaneous localization and mapping (SLAM) technology harness algorithms and sensory data to navigate an unfamiliar environment such as a brain, while tracking surgical instruments within that environment in real-time. It acts like a GPS that detects traffic jams and roadblocks in advance, giving doctors X-ray vision during surgery. This helps doctors accurately observe the location of lesions and tissues or blood vessels that need to be protected from multiple angles without the installation of the head clamp.

Saving More Lives

Neurosurgery is an area of medicine where AI technologies drive innovation. Experts predict that ophthalmology, ear, nose and throat (ENT), and hospital emergency departments can also benefit from the development of surgical navigation systems. The system can also be used for clinical teaching to improve the capacity of local medical care services.

By integrating computer vision, machine learning and deep learning with medical science, Tencent AI Lab continues to develop technologies that help doctors improve disease screening, diagnosis, and treatment, and ultimately, save more lives.