303
3C TecnologÃa. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Abril 2020
ADVANCES IN AUGMENTED REALITY (AR) FOR MEDICAL
SIMULATION AND TRAINING
Vladimir Ivanov
Herzen State Pedagogical University of Russia
Saint-Petersburg, (Russia).
E-mail: voliva@rambler.ru ORCID: https://orcid.org/0000-0001-8194-2718
Alexander Klygach
Herzen State Pedagogical University of Russia
Saint-Petersburg, (Russia).
E-mail: voolf00@yandex.ru ORCID: https://orcid.org/0000-0002-2984-0201
Sam Shterenberg
Pavlov First Saint-Petersburg State Medical University
Saint-Petersburg, (Russia).
E-mail: sam.d.s@mail.ru ORCID: https://orcid.org/0000-0002-6428-8328
Sergey Strelkov
Herzen State Pedagogical University of Russia,
Saint-Petersburg, (Russia).
E-mail: sergin3d2d@gmail.com ORCID: https://orcid.org/0000-0002-4830-5407
Jason Levy
University of Hawaii,
Honolulu, (USA).
E-mail: jlevy@hawaii.edu ORCID: https://orcid.org/0000-0002-9978-5412
Recepción:
26/02/2020
Aceptación:
17/04/2020
Publicación:
30/04/2020
Citación sugerida Suggested citation
Ivanov, V., Klygach, A., Shterenberg, S., Strelkov, S., y Levy, J. (2020). Advances in augmented reality
(AR) for medical simulation and training. 3C TecnologÃa. Glosas de innovación aplicadas a la pyme. Edición
Especial, Abril 2020, 303-312. http://doi.org/10.17993/3ctecno.2020.specialissue5.303-312
304
http://doi.org/10.17993/3ctecno.2020.specialissue5.303-312
3C TecnologÃa. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Abril 2020
ABSTRACT
Digital technologies are transforming the eld of medical training, simulation and mode-
ling. Advances in the eld of virtual Augmented Reality (AR) and virtual simulation are
described in detail, particularly as they relate to medical education and training. An over-
view of key medical simulation tools is provided in order provide foundational knowledge
about this rapidly growing eld. A timely and valuable original Augmented Realty system
is put forward. The key components of this original system for medical training and simu-
lation include the following three dimensions: advances in open surgery, realistic visualiza-
tions and innovative haptic was used. Each component of this Augmented Reality system is
described in detail. First, the open surgery module emphasized appendectomies (the most
common surgical procedures used in our model). Second, three dierent approaches for
creating realistic and accurate 3D medical models were put forth. Third, haptic feedback
involved the use of an enhanced Novint Falcon system in which a custom grip provides
additional degrees of freedom. Finally, advances in game simulation, modeling and role
playing are discussed for the eld of emergency medicine.
KEYWORDS
Surgical Simulator, Virtual Reality, Real-time Rendering, 3D Visualization, Haptic Feed-
back, Open Surgery, Laparoscopy, Emergency Medicine, Simulations.
305
http://doi.org/10.17993/3ctecno.2020.specialissue5.303-312
3C TecnologÃa. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Abril 2020
1. INTRODUCTION
Medical simulations, modeling and visualizations have undergone a rapid shift the begin-
ning of 20th century due to a number of reasons (Kron et al., 2010). First, modern approa-
ches for less invasive surgery have redened surgical procedures such as endoscopy and ro-
botics surgery. Second, the dramatic rise of computing power has provided an opportunity
to implement complex simulations in real-time. Finally, more accurate algorithms for rigid
and soft body simulations, realistic 3d visualizations, haptic controllers, and virtual reality
have allowed medical simulation to be used for digital gaming rather than simply physical
modeling.
Specic approaches and technologies for medical simulations for medical simulations have
grown by leaps and bounds. For example, innovative research has occurred dealing with
the generation of textures of irregular objects from models and photo sequences (Chen et
al., 2003). The role of medical simulations has rapidly expanded throughout the healthcare
eld (Kunkler, 2006). This paper involves a case study of medical simulations for lap.
2. MODERN MEDICAL SIMULATION MARKET
2.1. MARKET OVERVIEW
According to Prescient & Strategic Intelligence data, the global surgical simulation market was
valued at $254.7 million in 2017 with a growing trend. The value of this eld is forecasted to
increase to twice its value in 2023 (Figure 1). Another notable trend is that augmented reality
(AR) and virtual reality (VR) are being used to enhance the quality and eî™»ciency of medical
training. Thus, it is expected that this market will continue to grow, and digital technologies
will continue to have a major impact on the medical simulation eld.
Figure 1. Worldwide surgical simulation market by offering (2013-2023).
306
http://doi.org/10.17993/3ctecno.2020.specialissue5.303-312
3C TecnologÃa. Glosas de innovación aplicadas a la pyme. ISSN: 2254 – 4143 Edición Especial Special Issue Abril 2020
2.2. MEDICAL SIMULATORS
VirtaMed is a company primarily focused on simulator development for orthopedics,
genecology and urology. The company develops surgical simulators which are designed
on a single exible plat-form with the ability to expand and add additional procedures. All
simulators are combined with an anatomical model to provide the optimal tactile feedback
and real-world manipulations. In addition, for better eî™»ciency each virtual procedure
allows for guided training: specic colored hints and ghost tools show trainees how to
perform dierent tasks (Figure 2).
Figure 2. ArthroS Ankle by VirtaMed AG.
NeuroVR is a platform for neurological training that enables neurosurgeons to practice
skills with the help of virtual reality (Figure 3). Such a system does not depend on real life
models but uses haptic controllers for VR manipulations. The range of allowed exercises
are derived from actual patient images, which provides more realistic and accurate images
of surgical procedures. The system also captures objective metrics and measures the
prociency of procedures in order to track educational progress.
Figure 3. NeuroVR system with stereoscopic microscopic view.
SurgicalScience is a company which develops various simulation products, mostly for
laparoscopy and endoscopy. The LapSim product is designed to improve psychomotor