Virtual reality is already being used in many industries. VR technology makes it possible to simulate surgeries and virtually work with patients' fears. More and more new applications are finding VR and AR in medicine, and we can look forward to how they will affect our healthcare system in the future. Could virtual reality revolutionize medicine? Will our visit to the doctor or hospital stay look completely different shortly than today?
We put VR to the test to answer these questions: Do the medical use cases show where the industry is headed, or are they just research projects with no prospects? Can VR and AR establish themselves as permanent components in healthcare? Next, we'll explore these questions by elaborating on the individual areas of virtual reality, augmented reality, and mixed reality and exploring their potential for the medical sector.
1. Virtual Reality: new possibilities for visualizing anatomy
Modern training for medical professionals thanks to VR technology
VR technology offers many possible applications for training medical professionals. Instead of using static two-dimensional images, anatomy can now be taught visually by 3D Visualization. Physiological principles, as well as the internal processes of the human body, can be visualized. In Virtual Reality, students can not only see organs and tissues, but they also have the opportunity to interact with them in a completely new way. Individual organs can be viewed from all sides and rotated 360 degrees. In this way, virtual reality technology allows for a journey inside the human body. Medical students need to get as much hands-on experience during training without compromising patient safety. Using VR HealthCare meets these requirements. It allows students to see the human body in a new way. Individual surgical steps and techniques can be practiced and repeated as often as necessary. In this way, the skills of medical students are specifically trained and refined.
One example of a possible virtual reality application is the installation of an artificial hip in a virtual operating room. Here, the VR simulator simulates the forces acting on the physician during the milling of the acetabulum. To create a realistic situation, the necessary physician assistants, surgical assistants, and necessary surgical instruments are also located in the virtual operating room. The VR simulator allows the simulation of different patient types, bone densities, approaches to the hip joint, several surgical instruments, and the virtual practice of various crises.
Data visualization for OR planning
In addition to training medical students, VR technology also helps plan surgeries. Before surgery, doctors receive a large amount of data and information that needs to be competently displayed and mapped out. In Virtual Reality, the data can be combined with images and portrayed realistically. Organs and tissues to be treated are visualized in three dimensions for the operating physician, who can view them from all sides and rotate them as they see fit.
To provide surgeons with as realistic an image as possible of the body part to be operated on, researchers at the Department of Biomedical Engineering at the University Hospital Basel and the University of Basel have developed a process that converts two-dimensional images from CT scans into three-dimensional virtual spaces in real-time. It allows doctors to see the joint being operated on in three dimensions at any angle and desired magnification. With Virtual Reality, the doctor can get a very intuitive sense of his options during surgery.
Transferring knowledge through virtual data rooms
Virtual worlds also allow for knowledge transfer. For example, experts from other cities and countries can join virtual data rooms, enabling information sharing.
One example of this is tumor boards, where experts from different fields work together to find the best therapy procedure for a patient.
Fighting Phobias in Virtual Worlds
Virtual reality also offers many possible applications in the treatment of patients. The properties of virtual reality are particularly suitable for treating phobias, an area in which a breakthrough has already been made. In the virtual world, the patient can face his fears without fear. He can learn to control his emotions and reactions and cope with his fears.
One example is the Virtual Reality Medical Center in the United States, which specializing in treating phobias using VR technology. Here, phobias such as fear of flying, fear of heights, claustrophobia, arachnophobia, panic disorder, or post-traumatic stress disorder are treated with the help of virtual reality. Patients learn to control their fear in virtual worlds. Treatment proceeds in stages. In this way, the patient slowly and gradually gets to know their anxiety in the virtual world. VR is excellent for treating phobias because it can be adapted to the needs and progression of each patient.
Virtual phobia treatments are also being researched in London. The Institute of Psychiatry at King's College, for example, is studying paranoia therapy. Here, patients are put into a virtual bar where they have to interact with avatars. The avatars are intentionally artificial, but psychologists communicate with their patients through them. In the background, other bar patrons talk about you or make fun of the patient. The goal is to prepare the patient for real-life situations and overcome his social fears.
2. Augmented reality: how tablets find their way into the operating room
Visualization and navigation during surgery
With AR technology, data can be displayed on a tablet, data viewing glasses, or projector. In medicine, calculated images of vessels or organs, such as the heart, can be made available to the surgeon. Optical tracking systems allow patient information and live pictures of the body to be transmitted to monitors in the operating room. The doctor can figure out where to start to achieve the best possible result with this information.
Computer scientists at the Technical University of Munich are also working on augmented reality in medicine. Together with surgeons at the University Hospital for Surgery, they develop a computer-based imaging and navigation system for minimally invasive surgery. During minimally invasive procedures, the surgeon has to rely on screens next to the operating table that shows him images from an endoscope camera and thus guides him through the patient's body. The doctor has to constantly move his head back and forth between the screen and the patient during surgery. The research team's goal is to use augmented reality to facilitate work during minimally invasive surgeries by overlaying information directly into the surgeon's field of vision.
Application in oncology surgeries
Fraunhofer researchers have also developed a navigational aid based on augmented reality technology that can be used in cancer surgeries. In the case of malignant tumors, it is necessary to determine the exact location of metastases to be able to remove them altogether. The 3D ARILE system is designed to help determine the precise location of a lymph node. It uses AR glasses with powerful software, near-infrared cameras, and indocyanine green fluorescent dye. The diseased lymph node becomes visible due to the fluorescent dye. The near-infrared cameras capture the fluorescence and reconstruct the diseased lymph node 3D. Its position is displayed in real-time on the surgeon's AR glasses screen. It allows him to see if he has successfully removed all the diseased tissue.
3. Mixed Reality in Medicine
HoloLens as a learning tool
Many medical facilities rely on HoloLens and use holograms to train medical professionals or even to plan and perform surgeries. Examples of the use of mixed reality include medical simulators such as an ultrasound simulator and a birth simulator.
An ultrasound simulator allows medical professionals to anatomically depict body parts, zooming in and rotating them as desired. The simulator can monitor fluoroscopy of the human anatomy in real-time.
The birth simulator can be used to teach all stages of childbirth and emergency scenarios.
Using 3D holograms, doctors can see the unborn baby and trace how it moves through the birth canal. As part of the training application, the challenge is also to address shoulder dystocia, a complication during natural childbirth.
Visualization of operations using holograms
In addition to training, Mixed Reality also has applications in surgery. Holograms help doctors prepare even more intensively for an upcoming surgery and allow for better patient education. Mixed reality is a valuable tool in surgical planning. At the Oslo University Hospital Intervention Center, surgical treatment for liver cancer and therapy for children with heart defects are complemented by the use of mixed reality. Thus, 3D models are created in advance to depict the liver, blood vessels, and tumors accurately. With this Visualization, the following steps can be planned more accurately, and the correct procedure for these operations can be determined.
During surgery, surgeons can use Mixed Reality to more accurately assess anatomical conditions and quickly locate the area to be operated on. Thus, one of the areas of Application is working with foci of epilepsy in the brain. These foci are usually only visible on MRI scans. However, it is impossible to distinguish between diseased tissue and healthy tissue during surgery. However, these foci must be removed entirely to prevent epileptic seizures permanently. Using mixed reality, surgeons can obtain images of the patient from MRI and CT scans, holographically displayed on HoloLens, thus determining which tissue is affected. Trauma surgeons can also take advantage of mixed reality, for example, when they need to reassemble shattered bones. With the HoloLens, the surgeon receives holographic images from the CT scan and can assemble the bones at the correct angle.
4. Conclusion
Our conclusion regarding VR, AR, or MR in medicine can be summarized as follows: Virtual Reality will revolutionize the industry, but it still has a long way to go. Let's be honest: The last time you visited your dentist or general practitioner, did you see any signs of VR? Probably not.
It reflects the state of the market: virtual reality, augmented reality, and mixed reality certainly offer fantastic possibilities. Because of its immersive nature and its ability to provide additional information, VR technology can be used successfully in the training of medical professionals and the planning and execution of surgeries. But: VR use cases are currently limited to the spectacular side of the healthcare sector: we are talking about surgeries, prominent institutes, or research institutions promoting virtual reality, augmented Reality, and Mixed Reality.
They are still a long way from widespread relevance. However, this can change quickly, as the past has shown (e.g., 3D printing in dentistry). However, several factors must come together for this to happen, which is also due to the unique structure and regulation of the market. Once technological innovations have been tested and accepted by the relevant decision-making bodies, associations and institutions, implementation can happen quickly. And before you know it, the next time you go to the doctor, the general practitioner will be standing in front of you with a tablet instead of a stethoscope.