Effectiveness of Virtual Reality in Nursing Education: Meta-Analysis

1With the rapid development of information technology and shortages of nurse workforce, a transformation of nursing education is needed to prepare nursing students for evolving and complex health care environments. In US nursing schools, 75,029 qualified applicants for bachelor’s degrees and nursing postgraduate courses were rejected in 2018 due to an insufficient number of faculty, clinical sites, classroom space, clinical preceptors, and budget constraints.

2The ultimate goal of nursing education is to promote the application of theoretical knowledge in clinical practice. However, limited clinical practice time affects the opportunity for students of having clinical experience with real patients. This lack of clinical practice, which prepares students for the real clinical environment, can contribute to nursing procedure errors that compromise the safety of patients. Narrowing the gap between theory and practice during the educational process is necessary, but poses several challenges to nursing educators. In this scenario, to guarantee the quality and safety of nursing education, educators have adopted various teaching strategies including simulation experience for students.

3Simulation has been shown to be a valuable teaching-learning strategy to support the changing world of nursing education and to help optimize the teaching process. As the National Council of State Boards of Nursing stated, simulation is a key component of nursing education. The use of simulation as a nursing education tool is becoming increasingly common, providing students with realistic opportunities to practice skills learned in theory. Through simulation, students have a variety of practical opportunities to repeat clinical scenarios and make immediate decisions and reflections.

4With the development of simulation technology, the virtual world was discovered—initially used in military and medical science and later, in medical education. Virtual reality (VR) is the use of computer technology to create an interactive three-dimensional (3D) world in which users have a sense of spatial presence. It provides a first-person active learning experience through different degrees of immersion, or, in other words, the real perception of the digital world and the ability to interact with objects and/or perform a series of actions in this digital world. VR is highly conducive to clinical and procedure-focused training by enabling simulation. VR simulation refers to the use of a variety of immersive, highly visual, 3D characteristics to replicate real-life situations and health care procedures, incorporating physical or other interfaces such as a computer keyboard, a mouse, speech/voice recognition, motion sensors, or haptic devices. Virtual simulation refers to the involvement of real people operating simulated systems via a computer screen (virtual, that is, as the situation is not physical or in real time), and may include surgical simulators used for on-screen procedural training, usually integrated with haptic devices to interact with the system. In general, VR can make simulation become an effective supplemental tool for teaching.

5As VR technology advances and becomes increasingly affordable, nursing education is being transformed. VR has gained increasing attention in the field of nursing education and been used to teach many nursing concepts including leadership, communication, decision-making, critical thinking, inclusivity, health appraisal, and disaster triage. The use of VR in simulations allows repetitive, hands-on training to develop cognitive and skill mastery among nursing students, which are usually defined as the measure of participants’ understanding of concepts and the ability of a participant to demonstrate a procedure or technique, respectively. Additionally, VR simulations can give nursing students the opportunity to practice skills in a safe environment without risk to patients. In a study, 98% of the participating students recommended virtual simulation for future use in nursing education.

6Although the use of VR has many advantages, some researchers have reported that VR is not more effective than other traditional methods on some outcomes such as knowledge and performance scores. There are still some inconsistencies on the effectiveness of VR among studies. Up to date, meta-analyses on the effectiveness of VR have been conducted in some areas of medicine and education. However, to the best of our knowledge, there is no meta-analysis evaluating the effectiveness of VR in nursing education. Only one article systematically reviewed and evaluated the effectiveness of VR without meta-analysis, focusing on the effectiveness of VR simulation compared to other simulated methods on clinical psychomotor skills for pre-registration nursing students. Therefore, there is a need to determine the effectiveness of VR in nursing education. The aim of this study was to perform a meta-analysis of the effectiveness of VR use on knowledge (participants’ understanding of concepts), skills (ability of participants to demonstrate a procedure or technique), satisfaction (participants’ perception with VR learning intervention), confidence (self-confidence in learning content and process), and performance time (time taken on the test task) in nursing education.

7This meta-analysis was conducted according to the PRISMA (Preferred Reporting Items for Systematic Review and Meta-Analyses) guidelines.

8An electronic literature search was carried out in the Cochrane Library, Web of Science, PubMed, Embase, and CINAHL (Cumulative Index to Nursing and Allied Health Literature) from their inception to December 2019. The search strategies used in PubMed, Embase, and the Cochrane Library are listed in the Multimedia Appendix 1. Slightly modified search strategies were used in the other databases. Additionally, we manually examined reference lists of the selected articles to retrieve other relevant publications. Two investigators conducted searches independently, and EndNote software was used to import and manage selected documents.

9This study included randomized controlled trials (RCTs) or trials employing quasi-experimental randomized design, including those in the form of dissertations and conference papers, based on the PICO (Population–Intervention –Comparison–Outcome) method. In this study, the PICO elements were as follows:

10Population: pre-/post-registration nursing students or nursing staff

11Intervention: all kinds of VR education methods

12Comparison: traditional education methods (including presentations, classes, written instructions, etc) and non-VR simulation methods (including high/low fidelity simulation, mannequin-based simulation, etc)

13Outcomes: knowledge, skills, satisfaction, confidence, and performance time

14Two reviewers (FQC and YFL) independently extracted information based on preset standards, including authors, publication date, nation, sample size, participants type, research project, intervention regimens, and outcomes.

15Two reviewers (FQC and YFL) assessed the studies’ quality independently by referring to the Cochrane Handbook for Systematic Reviews of Interventions, which includes 7 domains corresponding to a specific type of bias: (1) random sequence generation (selection bias); (2) allocation concealment (selection bias); (3) blinding of participants and personnel (performance bias); (4) blinding of outcome assessment (detection bias); (5) incomplete outcome data (attrition bias); (6) selective reporting (reporting bias); and (7) other biases. A judgement of “low risk,” “high risk,” or “unclear risk” of bias was assigned to each domain. When disagreements between reviewers could not be resolved through discussion, two additional reviewers (ZLS and JFG) made the final decision.

16The meta-analysis was conducted using RevMan 5.3, a desktop version of Review Manager software used for Cochrane intervention and flexible reviews. For continuous data, we reported standard mean difference (SMD) with 95% confidence intervals. In each analysis, I2 was used to measure the statistical heterogeneity among studies. According to the values of P and I2, the fixed-effect model (P>.1, I2<50%) or random-effects model (0<P<.1, I2≥50%) were selected.

17A total of 2716 potential studies were identified from 5 databases (n=2712) and relevant references (n=4). After removing 1072 duplicates, the remaining articles were reviewed and those that did not meet the inclusion criteria were excluded. A total of 1644 articles were screened by title and abstract, of which 1581 articles were excluded. A total of 63 full-text articles were downloaded and assessed, from which 51 were excluded. Finally, 12 studies, including 821 participants, were selected for this study. A flow chart of the study selection process is presented in Figure 1.

18Studies included trials conducted in 7 countries: United States, Turkey, Canada, Korea, Singapore, Portugal, and China. Two trials adopted a 3-arm group design, while 10 trials used a 2-arm group design. Sample sizes ranged from 20 to 172 participants. In all trials, participants were nursing students, except for one study in which participants were nursing staff. Six of 12 trials compared VR education with traditional education, while the remaining trials compared VR education with other simulation types including fidelity manikin, mannequin-based simulation, and plastic model. The characteristics of the participants, intervention details, and outcome measures are presented in Table 1. Supplementary information of intervention in experimental and control conditions is shown in Multimedia Appendix 2.

19Based on the Cochrane criteria, a risk-of-bias assessment is presented in Figures 2 and 3. Four of 12 studies reported randomized methods in detail, while the remaining 8 trials did not provide the methods of sequence generation. None of the trials provided concealment methods, except for one that reported the use of anonymization. In all trials, no blind method was used on participants due to the particularity of the intervention methods. Two trials reported employing blinding of assessors. Additionally, 2 studies mentioned dropouts without detail on handling information.

20A total of 7 studies reported knowledge scores as the outcome. The results indicated that VR education can improve knowledge of participants more effectively than the control conditions (SMD=0.58, 95% CI 0.41-0.75, P<.001, I2=47%, Figure 4).

21A total of 5 trials used skills as the outcome measure. The results indicated that there was no significant difference between VR education and other education methods on skills enhancement (SMD=0.01, 95% CI –0.24 to 0.26), P=.93, I2=37%; Figure 5).

22A total of 4 articles reported participants’ satisfaction scores. Participants in VR groups showed no difference when compared to control groups (SMD=0.01, 95% CI –0.79 to 0.80, P=.99, I2=86%). High heterogeneity was found. The leave-one-out method was used to carry out sensitivity analysis, and the random-effects model was adopted. One trial caused significant heterogeneity, showing VR education is more satisfactory to participants than the control conditions (SMD=1.30, 95% CI 0.63-1.97, P=.001; Figure 6).

23A total of 4 studies reported confidence results and showed no statistical difference between VR education and other education methods (SMD=0.00, 95% CI –0.28 to 0.27, P=.99, I2=0%; Figure 7).

24Performance time was employed as an outcome measure in 3 trials. There was no significant difference between the experimental and control groups (SMD=–0.55, 95% CI –2.04 to 0.94, P=.47, I2=97%]. Heterogeneity in this outcome was high. Therefore, the random-effects model was used, and the sensitivity analysis was carried out by using the leave-one-out method. Nevertheless, heterogeneity remained significant even when removing one study at a time (Figure 8).

25This meta-analysis assessed the effectiveness of VR simulation methods in nursing education. We found that VR education methods can improve the knowledge of nursing students. However, there was no difference between VR and other education methods on the outcomes of skills, satisfaction, confidence, and performance time.

26A total of 12 trials with 821 participants were included in the meta-analysis. All studies used VR education as the interventions in experimental groups, and education methods in control groups including traditional education, high/low fidelity manikin, mannequin-based simulation, and plastic model. Among the 12 studies, 4 trials reported random sequence generation. Only 1 study described the allocation concealment; 2 studies reported the blindness of outcome assessment. In addition, blinded interventions of students and educators were not possible because of the particularity of the VR education method. In general, the overall risk of bias of the included studies was judged to be unclear due to lack of information.

27For the outcome of knowledge, VR education showed more effectiveness on nursing education than traditional education or other simulation education methods. A qualitative study on VR use in nursing education also concluded that, through the concrete experience of the virtual patient simulation and the reflection tool, students could understand what they were taught and how to utilize the new knowledge. Additionally, a previous study, which focused on virtual reality for health professions education, indicated that VR with higher interactivity showed more effectiveness for knowledge. These studies support the fact that an interactive learning environment encourages students to establish connections between concepts. Most of the studies included in our meta-analysis used interactive VR education methods, which could explain the results.

28Our results found no significant difference between VR education and other education methods for the outcome of skills, which seems to be in line with a previous systematic review. The review concluded that virtual reality groups performed comparably to simulation groups in skill performance scores and skill success rate. In our study, all the included trials that reported skills employed other simulation education methods in control groups. Similarly, we concluded that VR was not more effective in improving skills than other simulation methods in nursing education. A possible reason for these results is that there is a gap between completing virtual cases and real practice. Nursing skills learned on a virtual platform may not be transferable to real situations effectively because of the immaturity of VR technology.

29There was no significant difference on participants’ satisfaction between VR education and education methods in control groups. High heterogeneity was found. Through sensitive analysis, we found that 1 of the 4 included studies showed that VR was more satisfactory. In one trial conducted in 2012, some participants pointed out the immaturity of VR technology affecting users’ satisfaction. In contrast, 2 studies in recent years showed no difference between the 2 groups. Thus, we consider that participants’ satisfaction with VR education may vary according to technical conditions. Although in the 21st century nursing students had already shown high levels of usefulness, ease, and intention to use clinical VR simulation, VR is not widely used in nursing education. With the progress of technology, VR can better satisfy the users. However, further research is needed to confirm our results.

30The results in confidence indicated no difference between experimental and control conditions. VR could not enhance the confidence of participants more effectively than control conditions, which was consistent with a previous study from Korea. When VR was used for operation exercises, it was often necessary to use a mouse at the same time. Thus, the operation method is more difficult in VR when compared with other simulations such as the manikin.

31We also conducted a meta-analysis of performance time. The results suggested that VR was not more effective on reducing performance time than other educational methods. We found large heterogeneity among studies, even when a sensitivity analysis was conducted by using the leave-one-out method. The observed heterogeneity may be due to the different research designs of the selected studies, such as operation projects, VR devices, and education methods in control groups. One study on the effectiveness of VR endoscopy simulation training analyzed performance time with sufficient data and found no difference between VR and control groups; however, the quality of the evidence was very low. In contrast, a study conducted in clinical medicine found that VR can help operators shorten performance time. Therefore, more experiments are needed in the future to study the effectiveness of VR on performance time in nursing education.

32Our study has the following strengths. First, our study is the first meta-analysis assessing the impact of VR on nursing education. Second, to assess the effectiveness of VR education, we evaluated 5 outcome measurements—knowledge, skills, satisfaction, confidence, and performance time—which can probably provide reference for nursing education.

33There are also some limitations in our study. First, we only included articles published in English, which may affect the results of meta-analysis. Second, some of the included studies failed to provide the details of sequence generation, allocation concealment, and blinding methods. Third, we included 12 studies that have different interventions in control groups, which may cause significant heterogeneity among the studies.

34This meta-analysis provides a comprehensive evaluation of the use of VR on nursing education. We found that VR education methods can improve nursing students’ knowledge. However, for the outcomes of skills, satisfaction, confidence, and performance time, there seems to be no difference between VR and other education methods. In general, the use of VR should be considered to enhance knowledge and as a complement of other simulation strategies to improve the quality and safety of clinical practice. However, the heterogeneity and risk of bias among the included studies should be taken into consideration. Rigorously designed large-scale studies are required to further confirm the results in this review.