Grant: Discovery Grants Program - Individual
University of Alberta
$28,000.00
Eye Responses of Users in a Virtual Medical Training Environment
EF 2019-20
Résumé
The proposed research is addressing challenges posed by image-guided tele-manipulation, a situation where images of the work site are captured by video cameras. As an extension of this, computer-generated virtual images/messages may be superimposed to create an augmented reality (AR) to guide movements. Multiple sources of visual inputs in the AR environment have potentials to alter perception and manipulation behaviors of human operators as well as interfere with their decision-making process. We plan to conduct three lines of research to address human factor issues raised from AR, each with a well-defined objective. ***Objective 1 aims to investigate the eye-hand coordination pattern of human operators in the AR environments. We plan to track eye and hand motions of operators while performing a continuous task with multiple steps guided by eyes verses by AR. Specifically, we plan to investigate whether proactive eye movement (rapid eye shifting onto the target before the hand) is preserved in an AR environment. Proactive eye movement has been validated as a behavior marker associated with expert eye-hand coordination. Since AR has been frequently used for skills training, knowledge gained from users' proactive gaze behaviors in AR will provide insights for designing better virtual training systems. ***Objective 2 is motivated by the fact that AR can be registered to the direct vision (by eyes) or video stream (by video camera) at any timeframe. Inappropriate registration of AR has a potential to add cognitive loads to the operator due to the interference between multiple sources of visual inputs. Increasing cognitive loads can lead to detrimental consequences for safe performance as the cognitive capacity for each individual is restricted. In this portion of the research, pupil response will precisely monitor the operator's mental activities and efforts while performing a continuous task in the AR environment. We aim to determine the optimal timing, amount and type of AR employed in order to maximize task performance and minimize distraction.***Objective 3 aims to strengthen our understanding on the stereo-perception of the operator during image-guided procedures. We have developed a multiple-camera system capable of providing additional sources of visual information for the operator to judge the depth of the view of the surgical site. Previous results have shown that the operator frequently shifts their gaze among several screens which lead to stress. We plan to design a smart interface where an operator's gaze can be detected and used to automatically select the best viewing perspective for them without adding searching loads. In addition, by introducing a stereoscope (equipped with two video cameras) to the tele-manipulation, we plan to investigate whether operator can effectively process visual information for making correct judgments on the speed of moving objects and provide a timely response. **
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Alberta, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2016-06462
Name nom
Zheng, Bin
Department département
Medicine and Dentistry, Faculty of
Organization id
9
Institution établissement
University of Alberta
Province en
Alberta
Province fr
Alberta
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2019
Competition year année de concours
2016
Award amount
28000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1502
Committee name en
Biological Systems and Functions
Committee name fr
Systèmes et fonctions biologiques
Area of application code
1100
Area of application group en
Health, education and social services
Area of application group fr
Santé publique, éducation et services sociaux
Area of application en
Health, education and social services
Area of application fr
Santé publique, éducation et services sociaux
Research subject code
2716
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Virtual reality and related simulations
Research subject fr
Réalité virtuelle et simulations connexes
Installment
inst:4 - 5
Partie
4
Nb partie
5
Application title
Eye Responses of Users in a Virtual Medical Training Environment
Keyword
NSERC
Application summary
The proposed research is addressing challenges posed by image-guided tele-manipulation, a situation where images of the work site are captured by video cameras. As an extension of this, computer-generated virtual images/messages may be superimposed to create an augmented reality (AR) to guide movements. Multiple sources of visual inputs in the AR environment have potentials to alter perception and manipulation behaviors of human operators as well as interfere with their decision-making process. We plan to conduct three lines of research to address human factor issues raised from AR, each with a well-defined objective. ***Objective 1 aims to investigate the eye-hand coordination pattern of human operators in the AR environments. We plan to track eye and hand motions of operators while performing a continuous task with multiple steps guided by eyes verses by AR. Specifically, we plan to investigate whether proactive eye movement (rapid eye shifting onto the target before the hand) is preserved in an AR environment. Proactive eye movement has been validated as a behavior marker associated with expert eye-hand coordination. Since AR has been frequently used for skills training, knowledge gained from users' proactive gaze behaviors in AR will provide insights for designing better virtual training systems. ***Objective 2 is motivated by the fact that AR can be registered to the direct vision (by eyes) or video stream (by video camera) at any timeframe. Inappropriate registration of AR has a potential to add cognitive loads to the operator due to the interference between multiple sources of visual inputs. Increasing cognitive loads can lead to detrimental consequences for safe performance as the cognitive capacity for each individual is restricted. In this portion of the research, pupil response will precisely monitor the operator's mental activities and efforts while performing a continuous task in the AR environment. We aim to determine the optimal timing, amount and type of AR employed in order to maximize task performance and minimize distraction.***Objective 3 aims to strengthen our understanding on the stereo-perception of the operator during image-guided procedures. We have developed a multiple-camera system capable of providing additional sources of visual information for the operator to judge the depth of the view of the surgical site. Previous results have shown that the operator frequently shifts their gaze among several screens which lead to stress. We plan to design a smart interface where an operator's gaze can be detected and used to automatically select the best viewing perspective for them without adding searching loads. In addition, by introducing a stereoscope (equipped with two video cameras) to the tele-manipulation, we plan to investigate whether operator can effectively process visual information for making correct judgments on the speed of moving objects and provide a timely response. **