Virtual Touch and Visualization Tools for Surgery and Medicine

Virtual Rehearsal of Temporal Bone Surgery

Traditionally, surgeons prepare for an operation by studying image data, such as CT or MRI, in two-dimensions. Conceptualizing an accurate three-dimensional plan based on two-dimensional imaging is exceedingly difficult.


In collaboration with the team at Stanford Otolaryngology and the BioRobotics Lab, we have developed a surgical simulation platform that allows surgeons to both haptically and visually rehearse procedures on their patients’ specific anatomy before the operation. Volume-rendered, three-dimensional models can be viewed from any angle and can be manipulated in a surgically meaningful way, represented here by the use of a surgical drill.


A virtual procedure can be simultaneously followed via conventional tri-planar views for guidance, anatomical reference, and to monitor tissue removal. An endoscope can be used to visualize hard to reach or conceptually difficult anatomy, useful both pre-operatively and for training.


In conjunction with pre-operative surgical rehearsal, we have developed a virtual library of cases accessible to surgeons and trainees alike. A virtual case can be linked to the patient’s intra-operative surgical video, and can be followed for education and training.

Capture-VR-10   Capture-OR-10
Capture-VR-44b   Capture-OR-44b
Capture-VR-81   Capture-OR-81

Similarly a trainee may attempt a procedure and at any time overlay the same procedure performed by an expert, visually noting where the trainee has over- or under-dissected.


By way of segmentation, our simulator can represent difficult to comprehend anatomical spatial relationships while remaining specific to a particular patient’s anatomy. We can use pre-operative image data to create realistic, immersive anatomical and surgical training modalities, replacing traditional 2D anatomical textbooks.


In the hands of both novice and expert surgeons, we are currently validating our simulator to objectively measure performance enhancement obtained by pre-operative surgical rehearsal, ultimately leading to safer and more effective patient care.