Augmented reality navigation in neurosurgery
Augmented Reality (AR) is becoming a transformative tool in neurosurgery, enhancing both the precision and effectiveness of surgical procedures.
Ways AR is being integrated into neurosurgery
1. Preoperative Planning: AR can provide neurosurgeons with a 3D visualization of the patient’s anatomy. By overlaying images such as MRIs or CT scans onto the patient’s body in real time, AR helps surgeons better understand the location of tumors, blood vessels, or other critical structures. This improves the ability to plan the approach and minimize risks during surgery.
2. Intraoperative Guidance: During surgery, AR can project important data such as the location of deep-seated tumors, the path of nerves, or potential blood vessels directly onto the surgical field. This allows surgeons to navigate complex structures with greater accuracy, reducing the risk of damage to healthy tissue and improving patient outcomes.
3. Enhanced Visualization: AR systems can provide neurosurgeons with a real-time, detailed view of the surgical site, combining images from different imaging modalities (MRI, CT, etc.) and providing a more comprehensive understanding of the area. This enhanced visualization can be especially beneficial when performing procedures in areas of the brain or spine that are difficult to access or view.
4. Neurosurgical Training and Neurosurgical Education: AR is also being used for training and educational purposes. Neurosurgery residents and students can engage with interactive 3D models and simulations of human anatomy, improving their understanding of complex procedures without the need for live patient involvement. This is a useful tool for hands-on learning and can enhance both technical and decision-making skills.
5. Real-time Data Integration: Some AR systems are capable of integrating data from other sources, such as navigation systems, robotic devices, and electrophysiological monitoring. This integrated approach can help surgeons make more informed decisions and respond more quickly to any changes during surgery.
6. Postoperative Care: AR is being explored for post-surgery monitoring. It can potentially be used to visualize the healing process in real-time, providing surgeons with data on the recovery of tissues and any complications that might arise after the procedure.
Overall, AR in neurosurgery offers a promising future in terms of improving surgical accuracy, reducing complications, and enhancing both patient and educational outcomes.
Augmented reality has emerged as a promising training tool in neurosurgery. This is demonstrated in the wide range of cases in technical training and anatomic education. It remains unclear how AR-based training compares directly with traditional training methods; however, AR shows great promise in the ability to further enhance and innovate neurosurgical education and training 1)
Virtual reality simulators have been proposed as tools to understand, assess, and train neurosurgery residents 2) 3) 4) 5) 6).
An important element of simulator performance is the capacity of simulators to distinguish operator expertise. Most studies on operator performance have utilized “metrics.” 7) 8) 9) 10) 11) 12) 13) 14) 15) 16) 17).
Chan et al., highlights a selection of recent developments in research areas related to virtual reality simulation, including anatomic modeling, computer graphics and visualization, haptics, and physics simulation, and discusses their implication for the simulation of neurosurgery 18).
Medicine and surgery are turning towards simulation to improve on limited patient interaction during residency training. Many simulators today utilize virtual reality with augmented haptic feedback with little to no physical elements.
To optimize the learning exercise, it is essential that both visual and haptic simulators are presented to best present a real-world experience. Many systems attempt to achieve this goal through a total virtual interface.
Bova et al., approach has been to create a mixed-reality system consisting of a physical and a virtual component. A physical model of the head or spine is created with a 3-dimensional printer using deidentified patient data. The model is linked to a virtual radiographic system or an image guidance platform. A variety of surgical challenges can be presented in which the trainee must use the same anatomic and radiographic references required during actual surgical procedures.
Using the aforementioned techniques, they have created a ventriculostomy simulators, percutaneous radiofrequency trigeminal rhizotomy, and spinal instrumentation.
The system has provided the residents an opportunity to understand and appreciate the complex 3-dimensional anatomy of the 3 neurosurgical procedures simulated. The systems have also provided an opportunity to break procedures down into critical segments, allowing the user to concentrate on specific areas of deficiency 19).
Shakur et al., developed a real-time augmented reality simulator for percutaneous trigeminal rhizotomy using the ImmersiveTouch platform. Ninety-two neurosurgery residents tested the simulator at American Association of Neurological Surgeons Top Gun 2014. Postgraduate year (PGY), number of fluoroscopy shots, the distance from the ideal entry point, and the distance from the ideal target were recorded by the system during each simulation session. Final performance score was calculated considering the number of fluoroscopy shots and distances from entry and target points (a lower score is better). The impact of PGY level on residents' performance was analyzed.
Seventy-one residents provided their PGY-level and simulator performance data; 38% were senior residents and 62% were junior residents. The mean distance from the entry point (9.4 mm vs 12.6 mm, P = .01), the distance from the target (12.0 mm vs 15.2 mm, P = .16), and final score (31.1 vs 37.7, P = .02) were lower in senior than in junior residents. The mean number of fluoroscopy shots (9.8 vs 10.0, P = .88) was similar in these 2 groups. Linear regression analysis showed that increasing PGY level is significantly associated with a decreased distance from the ideal entry point (P = .001), a shorter distance from target (P = .05), a better final score (P = .007), but not number of fluoroscopy shots (P = .52).
Because technical performance of percutaneous rhizotomy increases with training, they proposed that the skills in performing the procedure in there virtual reality model would also increase with PGY level, if this simulator models the actual procedure. The results confirm this hypothesis and demonstrate construct validity 20).
Lemole et al., use the ImmersiveTouch (ImmersiveTouch, Inc., Chicago, IL) virtual reality platform, developed at the University of Illinois at Chicago, to simulate the task of ventriculostomy catheter placement as a proof-of-concept. Computed tomographic data are used to create a virtual anatomic volume.
Haptic feedback offers simulated resistance and relaxation with passage of a virtual three-dimensional ventriculostomy catheter through the brain parenchyma into the ventricle. A dynamic three-dimensional graphical interface renders changing visual perspective as the user's head moves. The simulation platform was found to have realistic visual, tactile, and handling characteristics, as assessed by neurosurgical faculty, residents, and medical students.
They developed a realistic, haptics-based virtual reality simulator for neurosurgical education. The first module recreates a critical component of the ventriculostomy placement task. This approach to task simulation can be assembled in a modular manner to reproduce entire neurosurgical procedures 21).
The virtual reality surgical training of thoracic pedicle screw instrumentation effectively improves surgical performance of novice residents compared to those with traditional teaching method, and can help new beginners to master the surgical technique within shortest period of time 22).
Augmented reality technology
Augmented reality technology has been used for intraoperative image guidance through the overlay of virtual images, from preoperative imaging study, onto the real-world surgical field.
The direct projection of a virtual image to the patients head, skull, or brain surface in real-time is an augmented reality system that can be used for Image-Guided Neurosurgery 23).
Information supplied by an image-guidance system can be superimposed on the operating microscope oculars or on a screen, generating augmented reality. Recently, the outline of a patient's head and skull, injected in the oculars of a standard operating microscope, has been used to check the registration accuracy of image guidance.
A commercially available image-guidance system and a standard operating microscope were used. Segmentation of the brain surface and cortical blood vessel relief was performed manually on preoperative computed tomography and magnetic resonance images. The overlay of segmented digital and real operating-microscope images was used to monitor image-guidance accuracy. Adjustment for brain shift was performed by manually matching digital images on real structures.
Experimental manipulation on a phantom proved that the brain surface relief could be used to restore accuracy if the primary registration shifted. Afterward, the technique was used to assist during surgery of 5 consecutive patients with 7 deep-seated brain tumors. The brain surface relief could be successfully used to monitor registration accuracy after craniotomy and during the whole procedure. If a certain degree of brain shift occurred after craniotomy, the accuracy could be restored in all cases, and corticotomies were correctly centered in all cases.
The proposed method was easy to perform and augmented image-guidance accuracy when operating on small deep-seated lesions 24).
Although setups based on augmented reality have been used for various neurosurgical pathologies, very few cases have been reported for the surgery of arteriovenous malformations (AVM).
5 patients underwent AVM resection assisted by augmented reality. Virtual three-dimensional models of patients' heads, skulls, AVM nidi, and feeder and drainage vessels were selectively segmented and injected into the microscope's eyepiece for intraoperative image guidance, and their usefulness was assessed in each case.
Although the setup helped in performing tailored craniotomies, in guiding dissection and in localizing drainage veins, it did not provide the surgeon with useful information concerning feeder arteries, due to the complexity of AVM angioarchitecture.
The difficulty in intraoperatively conveying useful information on feeder vessels may make augmented reality a less engaging tool in this form of surgery, and might explain its underrepresentation in the literature. Integrating an AVM's hemodynamic characteristics into the augmented rendering could make it more suited to AVM surgery 25).
Solves the problem of view switching in traditional image-guided neurosurgery systems by integrating computer-generated objects into the actual scene. However, the state-of-the-art AR solution using head-mounted displays has not been widely accepted in clinical applications because it causes some inconvenience for the surgeon during surgery.
The easy-to-use Tablet-AR system presented in a study is accurate and feasible in clinical applications and has the potential to become a routine device in AR neuronavigation 26).
Augmented reality technology has been used for intraoperative image guidance through the overlay of virtual images, from preoperative imaging studies, onto the real-world surgical field. Although setups based on augmented reality have been used for various neurosurgical pathologies, very few cases have been reported for the surgery of arteriovenous malformations (AVM).
The difficulty in intraoperatively conveying useful information on feeder vessels may make augmented reality a less engaging tool in this form of surgery, and might explain its underrepresentation in the literature. Integrating an AVM's hemodynamic characteristics into the augmented rendering could make it more suited to AVM surgery 27).
Although further studies need to be performed to evaluate whether certain groups of aneurysms are more likely to benefit from it. Further technological development is required to improve its user friendliness 28).
Head-mounted augmented reality surgical navigation
Indications
In minimally invasive spring-assisted craniectomy, surgeons plan the surgery by manually locating the cranial sutures. However, this approach is prone to error. Augmented reality (AR) could be used to visualize the cranial sutures and assist in the surgery planning. The purpose of our work is to develop an AR-based system to visualize cranial sutures and to assess the accuracy and usability of using AR-based navigation for surgical guidance in minimally invasive spring-assisted craniectomy.
An AR system was developed that consists of an electromagnetic tracking system linked with a Microsoft HoloLens. The system was used to conduct a study with two skull phantoms. For each phantom, five sutures were annotated and visualized on the skull surface. Twelve participants assessed the system. For each participant, model alignment using six anatomical landmarks was performed, followed by the participant delineation of the visualized sutures. In the end, the participants filled out a system usability scale (SUS) questionnaire. For evaluation, an independent optical tracking system was used and the delineated sutures were digitized and compared to the CT-annotated sutures.
For a total of 120 delineated sutures, the distance of the annotated sutures to the planning reference was [Formula: see text] mm. The average delineation time per suture was [Formula: see text] s. For the system usability questionnaire, an average SUS score of 73 was obtained.
CThe developed AR system has good accuracy (average 2.4 mm distance) and could be used in the OR. The system can assist in the pre-planning of minimally invasive craniosynostosis surgeries to locate cranial sutures accurately instead of the traditional approach of manual palpation. Although the conducted phantom study was designed to closely reflect the clinical setup in the OR, further clinical validation of the developed system is needed and will be addressed in a future work 29).
A MEDLINE search for “neurosurgery AND (simulation OR virtual reality)” retrieved a total of 1,298 articles published in the past 10 years. After eliminating studies designed solely for education and training purposes, 28 articles about the clinical application remained. The finding that the vast majority of the articles were about education and training rather than clinical applications suggests that several issues need be addressed for clinical application of surgical simulation. In addition, 10 of the 28 articles were from Japanese groups. In general, the 28 articles demonstrated clinical benefits of virtual surgical simulation. Simulation was particularly useful in better understanding complicated spatial relations of anatomical landmarks and in examining surgical approaches. In some studies, Virtual reality models were used on either surgical navigation system or augmented reality technology, which projects virtual reality images onto the operating field. Reported problems were difficulties in standardized, objective evaluation of surgical simulation systems; inability to respond to tissue deformation caused by surgical maneuvers; absence of the system functionality to reflect features of tissue (e.g., hardness and adhesion); and many problems with image processing. The amount of description about image processing tended to be insufficient, indicating that the level of evidence, risk of bias, precision, and reproducibility need to be addressed for further advances and ultimately for full clinical application 30).
see Augmented reality for pedicle screw insertion.
The use of intraoperative navigation during microscope cases can be limited when attention needs to be divided between the operative field and the navigation screens. Heads-up display (HUD), also referred to as augmented reality, permits visualization of navigation information during surgery workflow.
Mascitelli et al. retrospectively reviewed patients who underwent HUD-assisted surgery from April 2016 through April 2017. All lesions were assessed for accuracy and those from the latter half of the study were assessed for utility.
Seventy-nine patients with 84 pathologies were included. Pathologies included aneurysms (14), arteriovenous malformations (6), cavernous malformations (5), intracranial stenosis (3), meningiomas (27), metastases (4), craniopharygniomas (4), gliomas (4), schwannomas (3), epidermoid/dermoids (3), pituitary neuroendocrine tumors (2) hemangioblastoma (2), choroid plexus papilloma (1), lymphoma (1), osteoblastoma (1), clival chordoma (1), Cerebrospinal fluid fistula (1), abscess (1), and a cerebellopontine angle Teflon granuloma (1). Fifty-nine lesions were deep and 25 were superficial. Structures identified included the lesion (81), vessels (48), and nerves/brain tissue (31). Accuracy was deemed excellent (71.4%), good (20.2%), or poor (8.3%). Deep lesions were less likely to have excellent accuracy (P = .029). HUD was used during bed/head positioning (50.0%), skin incision (17.3%), craniotomy (23.1%), dural opening (26.9%), corticectomy (13.5%), arachnoid opening (36.5%), and intracranial drilling (13.5%). HUD was deactivated at some point during the surgery in 59.6% of cases. There were no complications related to HUD use.
HUD can be safely used for a wide variety of vascular and oncologic intracranial pathologies and can be utilized during multiple stages of surgery 31).
Head-mounted augmented reality surgical navigation
Prospective observational studies
Truckenmueller et al. in a prospective study assesses the acceptance and usefulness of augmented 360° virtual reality (VR) videos for early student education and preparation in the field of neurosurgery.
Thirty-five third-year medical students participated. Augmented 360° VR videos depicting three neurosurgical procedures (lumbar discectomy, brain metastases resection, clipping of an aneurysm) were presented during elective seminars. Multiple questionnaires were employed to evaluate conceptual and technical aspects of the videos. The analysis utilized ordinal logistic regression to identify crucial factors contributing to the learning experience of the videos.
The videos were consistently rated as good to very good in quality, providing detailed demonstrations of intraoperative anatomy and surgical workflow. Students found the videos highly useful for their learning and preparation for surgical placements, and they strongly supported the establishment of a VR lounge for additional self-directed learning. Notably, 81% reported an increased interest in neurosurgery, and 47% acknowledged the potential influence of the videos on their future choice of specialization. Factors associated with a positive impact on students' interest and learning experience included high technical quality and comprehensive explanations of the surgical steps.
This study demonstrated the high acceptance of augmented 360° VR videos as a valuable tool for early student education in neurosurgery. While hands-on training remains indispensable, these videos promote conceptual knowledge, ignite interest in neurosurgery, and provide a much-needed orientation within the operating room. The incorporation of detailed explanations throughout the surgies with augmentation using superimposed elements, offers distinct advantages over simply observing live surgeries 32)