Queen's University

Augmented reality image overlay for intraoperative surgical guidance

This project was supervised by Dr. Gabor Fichtinger from the School of Computing at Queen's University.

Augmented reality overlay systems can be used to project a CT image directly onto a patient during procedures. They have been actively trialed for computer-guided procedures, however, they have not become commonplace in practice due to restrictions of similar past systems. Such systems have not been handheld, and have had complicated calibration procedures. We put forward a handheld tablet-based system for assisting with needle interventions. The system consists of a tablet display and a 3-D printed reusable and customizable frame. A simple and accurate calibration method was designed to align the patient to the projected image. The entire system is tracked via camera, with respect to the patient, and the projected image is updated in real time as the system is moved around the region of interest. The resulting system allowed for 0.99mm mean position error in the plane of the image, and a mean position error of 0.61mm out of the plane of the image. This accuracy was thought to be clinically acceptable for tool using computer-guidance in several procedures that involve musculoskeletal needle placements. Medical professionals were asked to navigate patient images while using the image overlay to navigate patient images and plan needle insertions into a joint space. Participants completed a questionnaire regarding usability of the system in both handheld and table-mounted forms. The responses showed that all participants strongly agreed it was easy to learn how to use the image overlay system, and that it was easy to understand where the projected image was located on the patient. The image overlay system was designed and developed with the goal of bringing usable augmented reality guidance with CT and MR images to the patient’s bed-side to minimize patient discomfort. The system was found to satisfy accuracy requirements for a range of needle interventions, and in assessing the usability of the image overlay system, study participants identified the system as being simple to use and understand. In combination with these results and simple setup process, the image overlay system shows promise for use in clinical practice for image exploration and needle navigation.