Grant: Discovery Grants Program - Individual
University of Toronto
$22,000.00
Augmented Visual Displays for Teleoperation
FY 2017-18
Summary
The proposed research will focus on challenges posed by remote manipulation. In particular, we are considering situations in which a human operator is in some way ?removed? from a worksite and thus, in order to visualise the remote site and interact with it, must rely upon some kind of sensor ? such as cameras, endoscopes, microscopes, etc. Application domains that fall within this general class of teleoperation tasks include minimally invasive surgery, robotic surgery, remote training, military, space and underwater telerobotics and mining. Three related issues will be addressed, each with a defined objective. Objective A: To develop and test a stereoscopic augmented reality display capability that will enable one to present computer generated / virtual objects as if they lie underneath a real object surface, thereby creating the illusion of transparency . To achieve this, we shall exploit the conflict that occurs in visual perception between stereoscopic depth cues and interposition cues. The research will investigate the properties of a real object surface that facilitate this phenomenon, as well as how to artificially manipulate the apparent texture of such surfaces to achieve this. Our ultimate aim is to develop a flexible tool that will optimise this transition to transparency, ideally for any arbitrary real object surface. The other two issues relate to the ?keyhole effect?, which results from the limited field of view (FOV) typically provided by video sensors and which restricts the viewer?s ability to maintain awareness about information outside of the FOV. In earlier research we have demonstrated the usefulness of bifocal filtering for helping operators such as surgeons manipulate instruments outside the central magnified visual field, by providing a compressed image of that information. Objective B involves extending that concept to develop and test a polyfocal filtering capability, whose multiple foci will also be able to translate in real time in response to tracked instrument movements. Experimental testing of the concept will be based on surgically relevant three dimensional manipulation tasks. Objective C is motivated by the emergence of multiple camera endoscopes, which address the keyhole problem by providing multiple points of view. In our earlier work we demonstrated that real-time interactive view synthesis can provide valuable depth cues to an operator. In the present research we shall both improve the robustness of our viewpoint synthesis algorithms and carry out a more extensive evaluation of the concept using more complex surgically relevant tasks.
Program
Discovery Grants Program - Individual
Recipient
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2015-05304
Name nom
Milgram, Paul
Department département
Mechanical and Industrial Engineering - Mechanical and Industrial Engineering
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2017
Competition year année de concours
2015
Award amount
22000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1509
Committee name en
Civil, Industrial and Systems Engineering
Committee name fr
Génie civil, industriel et des systèmes
Area of application code
518
Area of application group en
Manufacturing processes and products
Area of application group fr
Procédés et produits industriels
Area of application en
Medical equipment and apparatus
Area of application fr
Appareils et instruments médicaux
Research subject code
1605
Research subject group en
Industrial engineering
Research subject group fr
Génie industriel
Research subject en
Human factors engineering
Research subject fr
Génie des facteurs humains
Partie
3
Nb partie
5
Application title
Augmented Visual Displays for Teleoperation
Keyword
NSERC
Application summary
The proposed research will focus on challenges posed by remote manipulation. In particular, we are considering situations in which a human operator is in some way ?removed? from a worksite and thus, in order to visualise the remote site and interact with it, must rely upon some kind of sensor ? such as cameras, endoscopes, microscopes, etc. Application domains that fall within this general class of teleoperation tasks include minimally invasive surgery, robotic surgery, remote training, military, space and underwater telerobotics and mining. Three related issues will be addressed, each with a defined objective. Objective A: To develop and test a stereoscopic augmented reality display capability that will enable one to present computer generated / virtual objects as if they lie underneath a real object surface, thereby creating the illusion of transparency . To achieve this, we shall exploit the conflict that occurs in visual perception between stereoscopic depth cues and interposition cues. The research will investigate the properties of a real object surface that facilitate this phenomenon, as well as how to artificially manipulate the apparent texture of such surfaces to achieve this. Our ultimate aim is to develop a flexible tool that will optimise this transition to transparency, ideally for any arbitrary real object surface. The other two issues relate to the ?keyhole effect?, which results from the limited field of view (FOV) typically provided by video sensors and which restricts the viewer?s ability to maintain awareness about information outside of the FOV. In earlier research we have demonstrated the usefulness of bifocal filtering for helping operators such as surgeons manipulate instruments outside the central magnified visual field, by providing a compressed image of that information. Objective B involves extending that concept to develop and test a polyfocal filtering capability, whose multiple foci will also be able to translate in real time in response to tracked instrument movements. Experimental testing of the concept will be based on surgically relevant three dimensional manipulation tasks. Objective C is motivated by the emergence of multiple camera endoscopes, which address the keyhole problem by providing multiple points of view. In our earlier work we demonstrated that real-time interactive view synthesis can provide valuable depth cues to an operator. In the present research we shall both improve the robustness of our viewpoint synthesis algorithms and carry out a more extensive evaluation of the concept using more complex surgically relevant tasks.