The comparison of teaching efficiency between virtual reality and traditional education in medical education: a systematic review and meta-analysis
Original Article

The comparison of teaching efficiency between virtual reality and traditional education in medical education: a systematic review and meta-analysis

Guanjie Zhao1#, Minjie Fan2#, Yibiao Yuan3, Fei Zhao4, Huaxing Huang1

1The First Clinical Medical College, Nanjing Medical University, Nanjing, China;2School of Pediatrics, Nanjing Medical University, Nanjing, China;3School of Basic Medical Sciences, Nanjing Medical University, Nanjing, China;4Department of Cardiothoracic Surgery, First Affiliated Hospital of Nanjing Medical University, Nanjing, China

Contributions: (I) Conception and design: All authors; (II) Administrative support: All authors; (III) Provision of study materials or patients: All authors; (IV) Collection and assembly of data: All authors; (V) Data analysis and interpretation: All authors; (VI) Manuscript writing: All authors; (VII) Final approval of manuscript: All authors.

#These authors contributed equally to this work.

Correspondence to: Huaxing Huang, MD. The First Clinical Medical College, Nanjing Medical University, 101 Longmian Avenue, Jiangning District, Nanjing 211166, China. Email: huanghuaxing@njmu.edu.cn; Yibiao Yuan, MD. School of Basic Medical Sciences, Nanjing Medical University, 101 Longmian Avenue, Jiangning District, Nanjing 211166, China. Email: yelloxiao@163.com; Fei Zhao, MD. Department of Cardiothoracic Surgery, First Affiliated Hospital of Nanjing Medical University, 300 Guangzhou Road, Nanjing 210029, China. Email: zhaofei@jsph.org.cn.

Background: Virtual reality (VR) technology has developed rapidly in recent years and has been applied in many fields, including medical education. A meta-analysis was performed to compare the examination pass rate of medical students educated using VR and those receiving traditional education to evaluate the teaching effect of VR in medical education.

Methods: The PubMed, Springer Link, Science Direct, and Wiley Online Library were searched from inception to May 2020. Articles meeting the inclusion criteria were then evaluated, relevant information extracted and a meta-analysis conducted. Students were allocated to a VR group, those trained using VR technology, and a traditional education group, those who received a traditional medical education.

Results: Six studies were included in the meta-analysis. The results indicate a significant difference between the pass rate of students educated using VR and those receiving traditional medical education. The odds ratios and confidence intervals of individual studies and our meta-analysis are illustrated with a forest plot.

Conclusions: Students in the VR group performed better than those in the traditional education group. Teaching with VR may enhance student learning in medical education. Medical schools should consider making greater use of VR when educating students.

Keywords: Virtual reality (VR); traditional education; medical education; meta-analysis

Submitted Mar 26, 2020. Accepted for publication Oct 12, 2020.

doi: 10.21037/atm-20-2785

Introduction

Traditional medical teaching is lecture-centric and didactic. This mode of teaching is largely based on attendance and memorization (1). In addition to theoretical learning, hands-on practical training also plays an important role in this teaching (2). These methods attract obvious limitations. Traditional lectures are boring and monotonous, and an absence of standardization and realistic models results in many students unable to master practical skills fully (2). Advances in digital technology have been proposed as offering a new way for modern medical education and training (3).

Virtual reality (VR) is a simulation in which computer-generated graphics are utilized to create a realistic world based on the user’s commands, such as gesture and verbal commands (4). An additional feature of VR is real-time interactivity, which means VR technology can receive time-changing input from the user and then modify the environment it created (5). This outstanding feature enables VR to have considerable potential in medical education at both the theoretical and practical levels.

In medical education, VR has been used successfully in the teaching of cranial anatomy allowing for the virtual manipulation of body parts, including the cranial bones by students wearing specialized goggles (6-8). VR has also been used to simulate surgical procedures. After receiving a specifically designed VR curriculum, both novice and experienced surgeons showed a significant improvement in performing laparoscopic colorectal surgery, illustrating that such training is equally suitable across all learning stages (9). Another study showed that participants performed better in knot tying scores when using a 360-degree VR video than a 2D video teaching group (10). Similarly, a study of a VR training curriculum for ophthalmology showed that the median pre-course score was improved by using a VR curriculum (11).

Despite this, some researchers still express concerns about the use of VR training in medical education as it may be viewed as less effective than traditional means of learning (12). Feedback from a laparoscopic suturing training program using box trainer only or box trainer and VR simulation suggested VR simulation had little effect on the operation (13). Some researchers are also worried about the physical and mental problems raised by VR, such as VR sickness (14). We conducted a meta-analysis to assess the overall strength of evidence comparing the pass rate of students trained using VR and those trained using traditional teaching methods to gauge the effectiveness of VR in medical education. We present the following article following the PRISMA reporting checklist (15) (available at http://dx.doi.org/10.21037/atm-20-2785).

Methods

Literature search strategy

According to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (15). A systematic search of relevant articles in PubMed, Springer Link, Science Direct, and Wiley Online Library databases was conducted from inception to May 2020. The search terms were (“Medical education” OR “Medical learning” OR “Medical training”) AND “Virtual reality.” There was no restriction on the language of publication.

Also, the references of all retrieved articles were reviewed to identify all potentially related articles.

Selection criteria

Studies were deemed eligible if they met the following criteria:

  • The studies were cohort studies or case-control studies;
  • The studies conducted research on VR and traditional education training in medical courses;
  • The pass rate of the students in VR group and traditional education group were available;
  • The studies were published as full-length articles.

No restriction was imposed concerning the type and size of the population studied. Two reviewers independently selected the articles that met the above criteria. A disagreement was resolved by consensus.

Data extraction

For each included study, we extracted the following characteristics:

  • The last name of the first author(s);
  • The year of the publication;
  • The country of the study;
  • The study design;
  • The number of students enrolled and the pass rate of students from VR and traditional education courses.

Statistical analysis

Students were allocated to a VR group and a traditional education group. We analyzed the pass rate of students in the VR group and the traditional education group. The odds ratio (OR) and 95% confidence interval (CI) were used to present the statistical values derived from case-control studies’ efficacy analysis. We calculated a pooled estimate of ORs and 95% CIs using the random effect model (16). Also, we calculated the quantity I2 to describe the degree of heterogeneity (17) and conducted a subgroup analysis. We assessed the quality of included studies using the Newcastle-Ottawa Scale. A score of 9 points represents the highest quality of the study. If the corresponding category’s content was not mentioned in the articles included, no points were awarded for that category. All statistical assessments were 2-sided, and the significance level was defined as P<0.05. The STAT 12.0 software was used for the statistical analysis of studies.

Results

Search results and study characteristics

Figure 1 shows the selection process to determine eligible studies. Several studies were excluded, either because they were qualitative or because they lacked a traditional education control group. In total, six studies (10,18-22) met our selection criteria and were included in the analysis (Table 1). These studies included a total of 633 experimental subjects involving first-year students, postgraduate, and hospital residents. Information on the total number of participants for each study and pass rate was extracted. Four studies (18,20-22) were conducted in North American countries (America and Canada), 1 (10) in England, and 1 (19) in Korea.

Figure 1 An illustration showing how the studies included in this meta-analysis.
Table 1
Table 1 Characteristics of studies included in the meta-analysis
Full table

Comparison of the pass rate of the VR group and traditional education group

The summary OR of pass rates for the VR group versus the traditional education group was 1.85 (95% CI: 1.32–2.58), with no statistically significant heterogeneity among the results of individual studies (I2=12.3%, P=0.34). The ORs and confidence intervals (CIs) of individual studies and our meta-analysis are illustrated with a forest plot (Figure 2). This indicates that students’ pass rate in the VR group was higher than those of students in the traditional education group.

Figure 2 Forest plot of the pass rate of the VR group versus the traditional education group. OR, odds ratio; CI, confidence interval; VR, virtual reality.

Comparison of the pass rate of the VR group and traditional education group within different countries

A subgroup analysis of the results from different countries showed an OR of 1.79 (95% CI: 1.23–2.60) in North American countries and 2.12 (95% CI: 0.98–4.60) in Asian and European countries (Table 2, Figure 3). This indicates that in North American countries, the pass rate of VR trained students are possibly higher than those of students educated in traditional courses. However, in Asian and European countries, this relationship was not significant.

Table 2
Table 2 Subgroup analysis of the passing rate of the VR group versus the traditional education group
Full table
Figure 3 Forest plot of the pass rate of the VR group versus the traditional education group according to different countries. OR, odds ratio; CI, confidence interval; VR, virtual reality.

Comparison of the pass rate of the VR group and traditional education group with different stages of medical training

A subgroup analysis of results in different stages of medical training showed an OR of 1.70 (95% CI: 0.69–4.23) in the freshmen group, 1.64 (95% CI: 1.13–2.39) in the postgraduate group, and 15.73 (95% CI: 2.35–105.04) in the hospital resident group (Table 2, Figure 4). This demonstrates significant differences in the pass rate of postgraduate and hospital resident groups. However, there was no significant difference in the pass rate in the freshmen group.

Figure 4 Forest plot of the pass rate of the VR group versus the traditional education group according to different professions. OR, odds ratio; CI, confidence interval; VR, virtual reality.

Comparison of pass rate of VR group and traditional education group with different sample sizes

We divided the sample size of groups used in the analysis into a small, medium, and large group. Groups with a sample size of 10 or less were designated as small; those with a sample size of greater than 10 and less than 100 designated as a medium; and those with a sample size of over 100 as large.

A subgroup analysis of these groups showed an OR of 13.50 (95% CI: 1.19–153.38) in the small group, 2.68 (95% CI: 1.32–5.44) in the medium group, and 1.53 (95% CI: 1.03–2.27) in the large group. This showed that regardless of the sample size, the pass rate of VR students was higher than that of those receiving traditional education (Table 2, Figure 5).

Figure 5 Forest plot of the pass rate of the VR group versus the traditional education group according to sample size. OR, odds ratio; CI, confidence interval; VR, virtual reality.

Discussion

The results of this meta-analysis suggest that students training with VR achieve better pass rates than those educated using traditional training methods. Our result showed a significant difference in the pass rate of the VR group and the traditional education group (OR =1.85, 95% CI: 1.32–2.58).

In the subgroup analysis stratified by profession, the pass rate of postgraduate and hospital residents in the VR group was higher than that of postgraduate and hospital residents in the traditional education group. This may suggest that VR can be used in training for the acquisition of complex skills and specialized knowledge. In the subgroup analysis stratified by country, VR training provided better pass rates indicating a global application. Results in the subgroup stratified by sample size produced similar results, indicating VR can train various student group sizes.

This analysis supports the results of other studies comparing the use of VR and traditional forms of medical training. VR training attracts strong student satisfaction concerning learning and knowledge acquisition (23-26). Students felt that VR improved their autonomic knowledge due to its real-time feedback and three-dimensional computer-generated scenery (4,6-8). VR has been used to simulate an operating room environment where students practiced laparoscopic colorectal surgery, resulting in a reduction in mistakes they made in a real environment (9). Students also express a greater level of confidence in completing the tasks they had been trained to do and developed a better understanding of the processes involved (27).

However, there are still some disadvantages to VR which remain to be solved. Firstly, the transfer of skills after VR training to the clinical setting needs further evaluation (28). Secondly, VR procedures, particularly those involving surgery, require careful design, including highly complex software design (29). Lastly, and most obviously, the question of whether VR can ever simulate a real-life environment completely accurately requires consistent review (2).

Our analysis’s strength was comparing the teaching effect of VR and traditional education based on the pass rate of examinations. The use of objective indicators, such as the pass rate, can effectively reduce bias. However, our study also had important limitations. The conclusions reached are less convincing, as only six studies were included in the analysis. Also, an important source of bias may have been introduced as the sample size of some of the analysis studies were small. Finally, the geographical coverage of the studies was limited.

Conclusions

This meta-analysis shows that there is a statistically significant difference in exam pass rates between medical students undertaking traditional and VR based education. The application of VR to medical education facilitates the acquisition of medical knowledge. Accordingly, consideration should be given to combining VR with traditional teaching. We contend that VR will play an important role in medical education in the future.

Acknowledgments

Funding: This work was supported by: High School Philosophy and Social Science Research Fund Project, 2019, Jiangsu Province (Grant number: 2019SJA0285); The Subject of the 13th Five-Years-Plan for Education Science, 2020, Jiangsu Province; Academic Works Publishing Support Project of Nanjing Medical University, 2019 (Grant number: 2019XZZ02).

Footnote

Reporting Checklist: The authors have completed the PRISMA reporting checklist. Available at http://dx.doi.org/10.21037/atm-20-2785

Peer Review File: Available at http://dx.doi.org/10.21037/atm-20-2785

Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/atm-20-2785). All the authors report grants from the Government of Jiangsu Province, China, and from Nanjing Medical University during the study. The authors have no other conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

  1. Kamei RK, Cook S, Puthucheary J, et al. 21st Century Learning in Medicine: Traditional Teaching versus Team-based Learning. Med Sci Educ 2014;22:57-64. [Crossref]
  2. Izard SG, Juanes JA, Garcia Penalvo FJ, et al. Virtual Reality as an Educational and Training Tool for Medicine. J Med Syst 2018;42:50. [Crossref] [PubMed]
  3. Kyaw BM, Saxena N, Posadzki P, et al. Virtual Reality for Health Professions Education: Systematic Review and Meta-Analysis by the Digital Health Education Collaboration. J Med Internet Res 2019;21:e12959. [Crossref] [PubMed]
  4. Li Y, Huang J, Tian F, et al. Gesture interaction in virtual reality. Virtual Reality & Intelligent Hardware 2019;1:84-112. [Crossref]
  5. Loureiro SMC, Sarmento EM, do Rosário JF. Managerial Challenges and Social Impacts of Virtual and Augmented Reality: IGI Global; 2020 [updated 2020. Available from: Available online: https://www.igi-global.com/chapter/incorporating-vr-ar-and-related-technologies-in-the-tourism-industry/248305
  6. Izard SG, Juanes Mendez JA, Palomera PR. Virtual Reality Educational Tool for Human Anatomy. J Med Syst 2017;41:76. [Crossref] [PubMed]
  7. Fang TY, Wang PC, Liu CH, et al. Evaluation of a haptics-based virtual reality temporal bone simulator for anatomy and surgery training. Comput Methods Programs Biomed 2014;113:674-81. [Crossref] [PubMed]
  8. Maresky HS, Oikonomou A, Ali I, et al. Virtual reality and cardiac anatomy: Exploring immersive three-dimensional cardiac imaging, a pilot study in undergraduate medical anatomy education. Clin Anat 2019;32:238-43. [Crossref] [PubMed]
  9. Beyer-Berjot L, Berdah S, Hashimoto DA, et al. A Virtual Reality Training Curriculum for Laparoscopic Colorectal Surgery. J Surg Educ 2016;73:932-41. [Crossref] [PubMed]
  10. Yoganathan S, Finch DA, Parkin E, et al. 360 degrees virtual reality video for the acquisition of knot tying skills: A randomised controlled trial. Int J Surg 2018;54:24-7. [Crossref] [PubMed]
  11. Saleh GM, Lamparter J, Sullivan PM, et al. The international forum of ophthalmic simulation: developing a virtual reality training curriculum for ophthalmology. Br J Ophthalmol 2013;97:789-92. [Crossref] [PubMed]
  12. Madan AK, Frantzides CT, Tebbit C, et al. Participants' opinions of laparoscopic training devices after a basic laparoscopic training course. Am J Surg 2005;189:758-61. [Crossref] [PubMed]
  13. Botden SM, Torab F, Buzink SN, et al. The importance of haptic feedback in laparoscopic suturing training and the additive value of virtual reality simulation. Surg Endosc 2008;22:1214-22. [Crossref] [PubMed]
  14. Kiryu T, So RH. Sensation of presence and cybersickness in applications of virtual reality for advanced rehabilitation. J Neuroeng Rehabil 2007;4:34. [Crossref] [PubMed]
  15. Moher D, Liberati A, Tetzlaff J, et al. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009;6:e1000097. [Crossref] [PubMed]
  16. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials 1986;7:177-88. [Crossref] [PubMed]
  17. Higgins JP, Thompson SG, Deeks JJ, et al. Measuring inconsistency in meta-analyses. BMJ 2003;327:557-60. [Crossref] [PubMed]
  18. Hashimoto DA, Petrusa E, Phitayakorn R, et al. A proficiency-based virtual reality endoscopy curriculum improves performance on the fundamentals of endoscopic surgery examination. Surg Endosc 2018;32:1397-404. [Crossref] [PubMed]
  19. Jung EY, Park DK, Lee YH, et al. Evaluation of practical exercises using an intravenous simulator incorporating virtual reality and haptics device technologies. Nurse Educ Today 2012;32:458-63. [Crossref] [PubMed]
  20. Maytin M, Daily TP, Carillo RG. Virtual reality lead extraction as a method for training new physicians: a pilot study. Pacing Clin Electrophysiol 2015;38:319-25. [Crossref] [PubMed]
  21. Park J, MacRae H, Musselman LJ, et al. Randomized controlled trial of virtual reality simulator training: transfer to live patients. Am J Surg 2007;194:205-11. [Crossref] [PubMed]
  22. Real FJ, DeBlasio D, Beck AF, et al. A Virtual Reality Curriculum for Pediatric Residents Decreases Rates of Influenza Vaccine Refusal. Acad Pediatr 2017;17:431-5. [Crossref] [PubMed]
  23. Yammine K, Violato C. A meta-analysis of the educational effectiveness of three-dimensional visualization technologies in teaching anatomy. Anat Sci Educ 2015;8:525-38. [Crossref] [PubMed]
  24. Silén C, Wirell S, Kvist J, et al. Advanced 3D visualization in student-centred medical education. Med Teach 2008;30:e115-24. [Crossref] [PubMed]
  25. Tworek JK, Jamniczky HA, Jacob C, et al. The LINDSAY Virtual Human Project: an immersive approach to anatomy and physiology. Anat Sci Educ 2013;6:19-28. [Crossref] [PubMed]
  26. Anderson SJ, Jamniczky HA, Krigolson OE, et al. Quantifying two-dimensional and three-dimensional stereoscopic learning in anatomy using electroencephalography. NPJ Sci Learn 2019;4:10. [Crossref] [PubMed]
  27. Wilson AS, O'Connor J, Taylor L, et al. A 3D virtual reality ophthalmoscopy trainer. Clin Teach 2017;14:427-31. [Crossref] [PubMed]
  28. Våpenstad C, Hofstad EF, Bo LE, et al. Lack of transfer of skills after virtual reality simulator training with haptic feedback. Minim Invasive Ther Allied Technol 2017;26:346-54. [Crossref] [PubMed]
  29. Wang Z, Liu Y, Luo H, et al. Is a Three-Dimensional Printing Model Better Than a Traditional Cardiac Model for Medical Education? A Pilot Randomized Controlled Study. Acta Cardiol Sin 2017;33:664-9. [PubMed]

(English Language Editors: B. Draper and J. Chapnick)

Cite this article as: Zhao G, Fan M, Yuan Y, Zhao F, Huang H. The comparison of teaching efficiency between virtual reality and traditional education in medical education: a systematic review and meta-analysis. Ann Transl Med 2021;9(3):252. doi: 10.21037/atm-20-2785

Article Options

Download Citation

Share