Grant: Discovery Grants Program - Individual
École de technologie supérieure
$24,000.00
Novel human-computer interaction paradigms for surgical augmented reality
FY 2024-25
Summary
The technology of augmented reality (AR) enables the superimposition of visual information onto real objects in our environment in order to facilitate the execution of certain tasks. By doing so, it is possible to radically improve the workflow in several work environments. Despite recent progress, AR is still largely absent from fields where precision, accuracy and safety are critical. The proposed research aims to remove the technological barriers to the adoption of AR for the specific application of computer-assisted surgery.
Every AR application can be characterized by its approach to three key processing steps: the rendering of 3D virtual models, the alignment of virtual models and real images and the blending of rendered models with reality. To spark adoption of AR for a specific application domain, the perceptual and physical accuracy of each processing step must be analysed and often, improved. In this research program, we will formulate novel human-computer interaction paradigms that take advantage of the user's expertise to address the accuracy of every processing step in surgical AR, an application domain where nearly all sources of inaccuracies are exemplified. More specifically, we will 1) use inexpensive virtual reality head-mounted displays and hand tracking devices as a basis to design and implement new interactive methods that facilitate the exploration of 3D medical images (rendering component), 2) propose interactive semi-automatic techniques to correct for the misalignment of real and virtual components of an AR image (aligning component) and, 3) design and implement rendering methods that enable the perceptually sound fusion of real and virtual components of an AR image (blending component).
At the end of the research program we will obtain a series of solutions to improve the physical and perceptual accuracy of each of the processing steps involved in the creation of AR images to guide surgical operations. More importantly, we will have defined a framework to analyse and correct the most important usability issues when addressing a new application domain for AR.
Program
Discovery Grants Program - Individual
Recipient
Location
Quebec, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2020-05084
Name nom
Drouin, Simon
Department département
Génie logiciel et des technologies de l'information
Organization id
53
Institution établissement
École de technologie supérieure
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2020
Award amount
24000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
1211
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Medical and health sciences
Area of application fr
Sciences médicales et de la santé
Research subject code
1901
Research subject group en
Biomedical engineering
Research subject group fr
Génie biomédical
Research subject en
Biomedical technology
Research subject fr
Technologie biomédicale
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Novel human-computer interaction paradigms for surgical augmented reality
Keywords
augmented reality; virtual reality; image-guided surgery; human-computer interaction; medical image visualization; real-time rendering; human visual perception; assisted registration
Application summary
The technology of augmented reality (AR) enables the superimposition of visual information onto real objects in our environment in order to facilitate the execution of certain tasks. By doing so, it is possible to radically improve the workflow in several work environments. Despite recent progress, AR is still largely absent from fields where precision, accuracy and safety are critical. The proposed research aims to remove the technological barriers to the adoption of AR for the specific application of computer-assisted surgery.
Every AR application can be characterized by its approach to three key processing steps: the rendering of 3D virtual models, the alignment of virtual models and real images and the blending of rendered models with reality. To spark adoption of AR for a specific application domain, the perceptual and physical accuracy of each processing step must be analysed and often, improved. In this research program, we will formulate novel human-computer interaction paradigms that take advantage of the user's expertise to address the accuracy of every processing step in surgical AR, an application domain where nearly all sources of inaccuracies are exemplified. More specifically, we will 1) use inexpensive virtual reality head-mounted displays and hand tracking devices as a basis to design and implement new interactive methods that facilitate the exploration of 3D medical images (rendering component), 2) propose interactive semi-automatic techniques to correct for the misalignment of real and virtual components of an AR image (aligning component) and, 3) design and implement rendering methods that enable the perceptually sound fusion of real and virtual components of an AR image (blending component).
At the end of the research program we will obtain a series of solutions to improve the physical and perceptual accuracy of each of the processing steps involved in the creation of AR images to guide surgical operations. More importantly, we will have defined a framework to analyse and correct the most important usability issues when addressing a new application domain for AR.