Virtual Reality in Neurosurgery: Beyond Neurosurgical Planning

1Virtual reality (VR) and augmented reality (AR) have intuitively found utility in neurosurgical planning, considered the increasing availability of papers available on this subject. However, the utility of VR and AR beyond neurosurgical planning remains unexplored and, therefore, opens avenues to expand their applicability. We intend to provide an overview of this vast expanse, beyond mere neurosurgical planning.

2VR involves the user being placed in a system that occludes the natural world and generates a virtual world that the user can experience. Depending on whether the virtual world is designed as a virtual environment or convincing substitute for the real world, VR is further classified as non-immersive and immersive, respectively. AR differs from VR in that there is a fusion of the natural and virtual world elements in AR, with superimposition of virtual images (as holograms) over the real-world environment, allowing for the simultaneous perception of (and, therefore, the interaction between) the projected virtual image and natural environment. This immersion and interaction were initially achieved by placing the user within active stereoscopic glasses (for immersion) and providing controllers that allowed for interaction and aided immersion with haptic feedback. Modifications to counter the smaller field of view afforded by the glasses, restricted area of work, and absence of head tracking (that allows for perspective change by the user) was made over time. However, these modifications were jarringly unnatural when the virtual image/environment and user’s perspective were misaligned. The addition of tracking the user and providing this feedback to the system allowed for a seamless user–environment interface. VR has gradually gained importance in neurosurgery, as illustrated in the article by Tomlinson et al. of VR-based training modules, which enhanced the understanding of neuroanatomy using these systems. Subsequent studies have focused on the utility of VR and AR in neurosurgical planning, drawing disproportionate focus to this one application. We aim to present an updated overview of the concepts, as well as the utilization, implications, and future possibilities of VR in neurosurgery, beyond the bounds of planning alone. Though our present review is not systematic, we have systematically searched and synthesized the review to reach meaningful conclusions. This review focuses on the role of VR in neurosurgery training and simulation, intraoperative utility, neuronavigation, postoperative rehabilitation, pain management, and future challenges.

3Though the present review is not intended to be a systematic review of the literature, we systematically conducted our search to remove redundancy and be more comprehensive in our narrative review. Systematic review for VR in neurosurgery, beyond planning, is not feasible, owing to limited homogenous studies. The lack of sufficient studies exemplifies the infancy of this aspect of VR and AR to conduct a systematic review. To construct this narrative review, we searched PubMed, PubMed Central, Google Scholar, the Scopus database, the Web of Science Core Collection database, and the SciELO citation index, from 1989–2021. Two authors (RM and HD) independently reviewed the retrieved studies, with title and abstract for further detailed review. An example of search strategy used in PubMed Central is as: “Virtual reality” [All Fields] AND (“neurosurgical procedures” [MeSH Terms] OR (“neurosurgical” [All Fields] AND “procedures” [All Fields]) OR “neurosurgical procedures” [All Fields] OR “neurosurgery” [All Fields] OR “neurosurgery” [MeSH Terms]). The search was started on 1 November 2020 and finished on 25 August 2021. Title and abstract were screened to identify articles providing discussion on the utility of VR, in addition to neurosurgical planning. Only articles in the English language were included. The full text of the articles was retrieved and reviewed from the eligible list to identify their inclusion to synthesize this review. In addition, the bibliography of the included articles was screened to identify other potentially eligible articles. We have excluded studies on neurosurgery planning and focused on studies exploring the use of VR in other scenarios in neurosurgery.

4Using the search strategy, we identified 487 (PubMed), 1097 (PubMed Central), 275 citations (Web of Science Core Collection database). This review is based on 95 references. We identified the utility of VR in neurosurgery, beyond planning, in the areas of neurosurgery training, neuronavigation, robotic neurosurgery, pain management, rehabilitation, and consent taking, as well as diagnostic tools (Table 1).

5Though VR was in use for panoramic viewing in the early eighteenth century, the first VR simulator was a flight simulator, invented in 1929, and the term “Virtual reality” was coined in 1987. The evolution of VR began with the induction of a young electrical engineer, Tom Furness of the United States Air Force. This insightful invention resulted in him being labelled the “The Godfather of Virtual Reality”. A VR system in medicine was first introduced by Robert Mann in orthopedics, followed by the induction of the head-mounted device (HMD) in the 1980s. VR was first used in the treatment of arachnophobia (fear of spiders) in 1998, and this remains the first documented use of this technology in the treatment of pathology. However, the first use of VR in the treatment of a neurosurgical disorder is relatively recent. This occurred in 2009 when David. B Clarke excised a Left frontal meningioma using the NeuroTouch neurosurgical simulator.

6Throughout the literature, VR has been used interchangeably to denote VR, AR, and even mixed reality (MR). However, VR, AR, and MR are fundamentally different technologies. VR refers to computed generated three-dimensional (3D) immersive environments, AR refers to the projection of computer-generated images onto real-world images, and MR refers to the projection of virtual objects into the real world, where the objects are spatially aware and responsive. Disadvantages of nausea, vomiting, temporarily impaired vision, and lack of sense of presence, noticed in the early usage, was due to the technical limitations of VR technology available at the time, as well as the inability of the human eye to fixate in-depth in a 3D-rendered image. Similar disadvantages during surgery have also been reported with the use of high definition 4K 3D exoscopes. This has been significantly addressed and improved with advances in technology and subsequent iterations of the concept.

7AR and VR facilitate navigation during complex and otherwise difficult neurosurgical interventions, in addition to their other potential functions. Two decades earlier, a report, published by neurosurgeons from Women and Brigham hospital, illustrated the potential advantage of using VR as a neuronavigation tool in 300 patients. The report consisted of using VR by constructing a 3D model of the patient’s head and fusing this with real-time MRI on a “double doughnut-MRI” or open magnet MRI. This facilitated the intraoperative navigation in minimally invasive surgeries by 3D construction of images preoperatively, an intraoperative fusion of these images with real-time MRI, and segmentation of the MRI, where essential landmarks and critical neurovascular structures were superimposed on the patient’s head on the screen. In the words of Grimson et al., “Virtual-reality technology is giving surgeons the equivalent of X-ray vision, helping them to remove tumors more effectively, to minimize surgical wounds and to avoid damaging critical tissues”. Volume graph-guided neuronavigation highlights the role of AR as a navigation tool for complex neurosurgical interventions by converting the operating field into a virtual operating tool. Microscope-assisted guided interventions (MAGI) provide 3D navigation, where preoperative 3D images are superimposed on the eyepieces of the operating microscope. The advantage of MAGI, as compared to the pointer-based navigation system, is that the surgeon does not have to shift his focus from the operative site to the screen; additionally, MAGI provides information regarding the location of nearby structures, as compared to the structures visualized only under the pointer in a pointer-based navigation system. Similarly, AR and VR are being used in image-enhanced endoscopy, where real and virtual images are superimposed with multiple layers to provide enhanced surgical exposure. Bichlmeier et al. introduced the concept of a virtual mirror in AR, similar to the utilization of the dental mirror, to visualize different aspects of the surgical territory. In a recent article, T. Fick et al. have shown that a stand-alone AR HMD can be used in lieu of conventional neuronavigation, with significant advantages, such as direct superimposition of images the patient, improved ergonomics, and diminishing attention shifts. The main benefit of AR HMD visualization in brain tumor surgery is the integrated continuous display, allowing for pointer-less navigation. Navigation view provides the highest usability, while blocking the operative field less frequently. Roethe et al. reported the use of the AR display in 44.4% of resection time, with a predominant AR type navigation view (75.7% of time), followed by target volume view (20.1% of time). Intraoperative VR has been described for critical mapping and discerning eloquent regions of the brain, especially in awake craniotomies. VR for awake craniotomy provides an immersive environment that is absent in traditional mapping techniques.

8Cerebrovascular surgery and neuro-interventional surgery rely heavily upon advanced neuroimaging techniques for operative decision-making and prognostication. Clinical application of VR can augment the diagnostic accuracy and efficacy of these techniques. Hybrid angio-suites enable neurointerventists to develop a VR immersive model, based on patient-specific anatomy for better procedural skill, training, and crisis resource management. Unique metric-based performance assessment, outside the angio suite, and ability to perform complex neuro intervention, including mechanical thrombectomy, with the same set of principles as in a live patient, has been a great utility of VR technology. Surgeons benefit from access to VR patient specific models too, for better diagnosing or planning management strategies, as well as for planning complex combined or hybrid procedures that require a combination of interventional and conventional surgical methods. This advantage holds true, irrespective of the surgeons existing skill set and years of experience.

9Another example of the clinical application of VR in diagnostics is to compare preoperative imaging with postoperative imaging, in order to identify junctional kyphosis in adult spinal deformity correction and eliminate its confounding effects on the sagittal alignment after the deformity correction procedure. Taken together, this indicates that, although VR may not supplant traditional imaging modalities in the diagnosis of disorders, it allows for expansion of the fidelity and accuracy of these measures.

10A detailed roadmap of neuroanatomy confers a higher degree of confidence and success in neurosurgical procedures. This guiding principle has led to several advancements in the last few decades, addressing the needs of neurosurgical training and simulation (Figure 1). A key challenge faced by the neurosurgeon-in-training is performing a procedure with bimanual-dexterity in a narrow corridor bound by complex and vital neurovascular structures and non-resilient bones. Neuronavigation is heavily relied upon by neurosurgeons-in-training for planning their approach and localization; however, it is not suitable for developing spatial reasoning abilities and an over reliance on neuronavigation paradoxically extends the development of skills. Moreover, as surgery progresses, the brain gradually shifts, which renders the navigation system, using preoperative images, less useful and accurate. To avoid this problem, an intraoperative brain imaging system (IBIS) was devised that identifies any discrepancy between intraoperative ultrasound and preoperative imaging. Using IBIS, the intraoperative stimulation is modified in real-time, and inaccuracies are updated using AR. VR, when used in simulation and training, proved to be a better alternative to reduce the cognitive load and operative stress duration, and proved to enhance efficacy for novice neurosurgeons. VR tools available for neurosurgical education and training include a multifunction head-mounted display (HMD: Microsoft HoloLens, Google Glass), haptic feedback (NeuroVR, Immersive touch, Procedicus Vascular Interventional System Trainer (VIST)), synthetic tissue simulator (Creaplast, SynDaver, iDU optics 3D-printed models, Thomas Jefferson university durotomy repair module), VOSTARS (video and optical see-through augmented reality surgical systems) HMD-based surgical navigation platform, and surgical planning devices (surgical theatre, Dextroscope, VPI reveal, Synaptive Medical). VR and computer simulation are used in areas of medicine, military, and pilot training to reduce the danger(s) involved by providing a virtual simulator and visual and haptic feedback. The challenge often faced is that physics-based simulators are computationally demanding and need resources, in terms of software and computing skills, to provide visual and haptic feedback, in addition to formal training in 3D immersive simulation. HMD are the most engaging displays in VR, others being google glass, consisting of an LED or OLED display with a high refresh rate (120 Hz) and latency time of 20 milliseconds or less. VR systems currently in use in neurosurgery training and simulation include part-task simulators (e.g., ventriculostomy catheter insertion models and subpial resection models), procedure simulation (NeuroTouch), surgical rehearsal platform (Dextroscope, virtual endoscopy), and robotic neurosurgery paradigms. Further, VR plays an essential role in tele-proctoring, in order to train surgeons on complex procedures, regardless of their geographical location. These immersive technologies impact global virtual connections, to empower low-and-middle-income countries and potential applications, in times of ongoing pandemics, such as COVID-19. Telemedicine will likely play an important role in future outpatient neurosurgery consultations, as this technology allows the surgeon to interact with the patient in a “merged reality” space, facilitating verbal, visual, and manual interaction between them. Certain quality control standards should be adhered to while using VR technology as a neurosurgical educational tool. These include, but are not limited to, high-resolution images, appropriate sound quality, haptic and visual feedback, high processing power, internet speed, the structure of the organs, and tissue fidelity. Key advantages of using VR technology in neurosurgery training, as compared to human and animal models, are its low cost, non-invasive nature, limitless repetition ability, and the sheer variety and diversity in cases that can be simulated. However, realism and resolution are an ever-present concern. The VR environment can be used to quantify surgeon performance, assess surgeon proficiency, and track training progression. Young surgeons demonstrated greater enthusiasm and enjoyment in learning with AR HMD visualization. Though more than 1000 articles were published on the use of VR in the medical sector, with significant numbers dedicated to neurosurgery, there are no controlled designed studies or high-quality studies with homogenous devices, thereby limiting the available evidence on the uses of VR technology in neurosurgery.

11Research and technical developments in spinal surgery exceed other surgical specialties with advancements in navigation, endoscopy, and the advent of robotic spinal surgery. However, the development of AR, VR, and MR in spine and orthopedic surgery is similar to brain surgery, in terms of cost-effectiveness, levels of recommendations, evidence, and lack of standardized measures. Common spinal procedures, for which simulation-based training is currently used, in order of frequency, are pedicle screw insertion, vertebroplasty, posterior cervical laminectomy, and foraminotomy. Leading simulator technologies in spinal surgery include Immersive Touch Simulator, Novint Falcon, Stealth 3D, and Osso VR, employing AR, VR, and MR techniques. Gradually, the world has witnessed a parallel increase in literature on VR in minimally invasive spine surgery, spine endoscopy, tumor management, and spinal deformity correction.

12In pain management, VR has initially been studied to treat trigeminal rhizotomy. Specifically, VR-assisted percutaneous radiofrequency trigeminal rhizotomy has shown safety and efficacy in studies and stresses the utility of virtual 3D CT scans in assessing the position and depth of the needle. VR has also shown efficacy in managing pain during minor procedures and other chronic illnesses. While this may not be strictly ‘neurosurgical‘, it is of importance in the ancillary management of patients during their stay in the neurosurgical service. Of particular importance in neurosurgery is that VR has shown to be efficacious in managing chronic pain that fails conventional opioid and physical therapy. This opens avenues in the utility of VR for chronic pain disorders in neurosurgery, such as failed back and regional specific pain syndromes.

13VR has been used in several novel ways, from creating a virtual environment to allow the patient to train in to the establishment of telerehabilitation services, in order to allow patients (who are understandably hard-pressed to travel) to pursue rehabilitation services at home, while their therapist can monitor progress from a different location. In addition to this, in an interesting article by Christiansen et al., VR was used to assess and quantify the level of cognitive deficit, following traumatic brain injury, by providing the patient with simple sequencing tasks. Nearly every rehabilitation domain has shown promise with the incorporation of VR, with examples including the assessment of driving with patients following trauma and providing leisure opportunities using video-based VR in young adults with physical and intellectual disabilities. Maresca et al. reported a positive impact on cognitive domains, motor recovery, balance improvement, and reduction in anxiety and depression in a 60-year old individual with incomplete cervical spinal cord injury, treated with combined traditional physiotherapy and VR rehabilitation system. Another prospective study reported similar results on balance improvement in spinal cord injury patients treated with a VR rehabilitation system.

14The advantages conferred by VR stem from the ability to gradually increase the complexity of tasks, facilitate simultaneous audio, visual, motor, and cognitive rehabilitation remain engaging and entertaining and offer positive, reward-based feedback, which are of paramount importance in preventing patient frustration and dropouts. Additionally, future strategies would be to make VR-based rehabilitation more universally accessible to promote patient compliance. When considering patients with long-term rehabilitative needs, VR and AR would be more economically viable, thereby alleviating the financial burden of an already distressed portion of society. In a seminal article, Weiss et al. poignantly observes that there is real-world improvement in patients that correlate with improvement in the virtual world, and sufficient research and optimization should be directed towards improving VR rehabilitation systems.

15The neuroArm leads this movement as an image-guided teleoperated neurosurgical robot. As with all new technologies, a significant amount of learning and relearning must occur before the operator gains sufficient proficiency. This creates a unique problem, as surgeons find themselves requiring training, in a short period of time (as dramatically new interventions have a latent lead time, followed by an exponential growth phase), with restricted access to the instrumentation being studied. The utility of VR, in allowing surgeons to train in the absence of readily deployed systems or time slots with those systems, has precedence in the field of laparoscopy. A more recent example is the pattern of adoption faced with the da Vinci Surgical System (Intuitive Surgical, Sunnyvale, California) and its companion VR trainer, the DV-Trainer (Mimic Technologies, Seattle, Washington). The same may be anticipated as robotic neurosurgery becomes more widely adopted. VR, in this interesting application, traces its roots back to its earliest widespread adoption, as similar to the field of avionics, ‘trainers’ can be developed designed on the same master console or interface used by the surgical robot. Added advantages are that, by designing the system on the same interface, the incorrect transfer of motor skills and haptic mismatch does not occur, as the development of a haptic interface that mimics an instrument is a difficult task. VR systems are additionally more cost-effective than additional robotic units, which require installation for training. Moreover, although there is a surge in robotic neurosurgery in different neurosurgical pathologies, most robotic neurosurgery advancement is limited to the computer interface between the surgeon and patient, further validating the use of VR trainers.

16An exciting vision of the future of robotics integrated with AR was presented by Madhavan et al., where a telerobotic system was fused with an AR overlay that allows for the dexterity and precision of the robotic system, fused with the added information (such as projected position(s) of vital structures and the pathology in question) from an AR overlay. Added functionality to the system can also be brought in using AR to measure distances and implants, obviating the need for conventional measuring devices and consolidating instrument diversity (which is especially advantageous in a robotic system).

17Informed consent (IC) is defined as “the communication process between a patient and physician that results in the patient’s authorization or agreement to undergo a specific medical intervention”. Neurosurgery is a high-risk specialty, with the highest rates of legal claims among all other specialties. In a recent randomized controlled trial, 33 patients were randomized 1:1:1 to three groups. In the two experimental groups, patients underwent 3D, immersive, informed consent with two different surgical planners (group 1 and group 2). In the control group, patients underwent an informed consent, supported by traditional 2D radiological images. All these patients were asked to fill the Spielberger state and trait anxiety self-evaluation questionnaire (state anxiety inventory (STAI Y-1) and trait anxiety inventory (STAI Y-2)), to assess individual situational and trait anxiety (higher scores reflect higher anxiety levels) after standard surgical consent and the planner(s) explained the surgical aspects and nuances using the Surgical Theater™ and Vesalius™ surgical planning platforms. The patients in the VR groups appreciated this communication experience, while their objective comprehension was higher (score mean (SD): group 1 82.65 (6.83); group 2 77.76 (10.19)), as compared with the control group (57.70 (12.49); p < 0.001). Although, the idea may be controversial, as providing extra information about surgical risks and complications may augment patients’ anxiety, especially when complications can be disabling. However, this avenue of VR utilization goes beyond regular neurosurgical planning and needs to be investigated significantly to reduce medicolegal risks and ensure patient understanding.

18VR has been utilized for training non-technical skills, such as communication, teamwork, and situational awareness in health care professionals. Extrapolating on this idea, VR and AR may, in the future, additionally find applicability in the training of surgical scrub nurses and operating room technicians, who require significant exposure before attaining the technical proficiency required of them. Although stated in the context of robotic assistants, the words of Laligam Shekar et al. state that “when observed, such surgeon–assistant teams appear to be as graceful as a ballet or symphony. But such a team takes time and active effort to develop”. The answer to this might lie in the field of VR. VR has also shown promise in mapping social cognition during awake craniotomy and other complex cognitive functions that are routinely not mapped during surgery. Katsevman et al. reported the use of a VR protocol as a feasible functional tool in awake-patient brain tumor surgery by using it as a complement during cognitive screening, in addition to language testing. Similarly, Mazerand et al. showed the use of intraoperative visual field assessment with a virtual reality headset during direct subcortical stimulation, to map the optical radiations and prevent a permanent visual field defect during awake surgery (a promising approach).

19Significant technical hurdles faced during awake surgery, such as mapping of the optic radiation, have also been overcome with the utility of VR.

20Finally, VR has also been used to improve patient compliance and comfort. Studies have shown the utility of VR in preparing patients for MRI examinations and VR headsets are commonly used to prevent the intense discomfort faced by patients while undergoing the MRI. These examples further indicate that the utility of VR and AR, which was initially limited by technological sophistication, is now inherently limited by artificially narrowed horizons.

21VR has faced challenges, predominantly due to the technical complexities in designing clinically valuable and relevant models. The main issues that have halted the monumental rise of VR in neurosurgery include, but are not limited to, feasibility in application and transference to real-life scenarios, ethics in supplementing with standard practices, and cost-effectiveness analyses. One of the critical drawbacks of VR application in neurosurgery, beyond presurgical planning, is the learning curve associated with it, when used to augment surgical procedures; this is much more significantly evident with the advancement of robotic neurosurgery. This steep learning curve in VR-assisted neurosurgery is an additional burden to the already time-consuming neurosurgical residency, which often requires further training and fellowships to acquire competence and expertise in specific neurosurgical procedures.

22The evaluation of neurosurgical performance, in which VR is used, is a newborn field of interest. This review has the feature of showing the different fields in which AR and VR are used in neurosurgery, as well as their possible implications in the next future. However, it has the limitations that, in order to limit the broad topic dealt and not disperse the information gathered in this review, tractography and white matter integrity in neuronavigation were not discussed.

23VR and AR show benefits in preoperative planning and multimodal neuronavigation for spine and brain surgery. In addition, the included studies suggested that VR and AR have beneficial effects for medical education and neurosurgical training. This paper reported the immense possibilities of VR in neurosurgery, beyond merely planning for surgical procedures. To generate relevant evidence in the next future, we need to rigorously evaluate AR and VR implementations, in order to better understand the strengths and limitations of HMD and other tools used during surgery, as well as in all fields of neurosurgery. The importance of VR and AR, especially in “social distancing”, in neurosurgery training for economically disadvantaged sections, prevention of medicolegal claims, and pain management and rehabilitation is promising and warrants further research.