Grant: Discovery Grants Program - Individual
École de technologie supérieure
$58,000.00
Study and development of immersive multi-sensory systems for avatar embodiment in virtual reality
FY 2024-25
Summary
Immersive virtual reality (VR) offers the possibility of simulating not only a user's surroundings but also his visual self-representation. In other words, the user sees himself as an avatar rather than seeing his actual body. When the avatar mimics the user's movements with a very short delay, the result is a subjective feeling of ownership of the avatar where the user feels the avatar is his actual physical body. When the virtual body is not aligned with the physical body, users experience a proprioceptive drift where the perceived position of his limbs drifts towards that of the virtual avatar. In recent years, our team has harnessed this illusion through the development of a VR system that has been shown to alter a user's movements through the real-time manipulation of the movements of his avatar. We have found that the user's brain signals can be used to measure the illusion of ownership and that the illusion can be created using vibrators that give the illusion of movement, rather than actual movements.
Our recent progress lead us to build upon this and take our research program to a new level in order to explore and apply the potential of real-time embodied avatars, progressing towards our ambitious long-term goal of developing a complete virtual surrogate body that is controlled and felt by its user. This research program consists of three threads. The 1st aims to integrate and extend our current technologies into a multisensory virtual body controlled by movement or imagined movements and felt through proprioceptive illusions. The 2nd aims to develop a system that allows for gradual real-time deformation of the avatar's upper and lower limbs (unilateral length, girth, segment proportions, etc.) during the embodiment experience in order to modify the user's body self-representation and alter motor control. The 3rd aims to combine a haptic exerciser with our self-avatar. Here the avatar will be driven by the movements and forces produced by users, in order to restore congruency between vision and motor control in post-stroke patients.
This research will lead to fully-embodied avatars that will be used in physical rehabilitation (to study and treat pathologies that cause motor impairment, such as stroke and spinal cord injury), sports (improving sport biomechanics and learning new tasks), telepresence systems (allowing a remote user to feel fully present), robotics (feeling the movements of the robot that is controlled) and full-body surrogacy (allowing paralyzed patients to control a virtual or robotic body). The research program will train 4 graduate students and 4 undergraduates, equipping them with skills that are at the cutting edge of technology and in high demand in Canadian industry. They will learn how to: develop complex multisensory VR systems; develop highly interactive video games and real-time computer animations; acquire, process and interpret biomedical signals; develop novel human-computer interaction (HCI) methods.
Program
Discovery Grants Program - Individual
Recipient
Location
Quebec, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2020-06171
Name nom
Labbe, David
Department département
Génie logiciel et des TI - Génie logiciel et des TI
Organization id
53
Institution établissement
École de technologie supérieure
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2020
Award amount
58000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2716
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Virtual reality and related simulations
Research subject fr
Réalité virtuelle et simulations connexes
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Study and development of immersive multi-sensory systems for avatar embodiment in virtual reality
Keywords
virtual reality; augmented reality; computer animation; multisensory biofeedback; motor control; biomechanics; electroencephalography; motor skills transfer; motor rehabilitation
Application summary
Immersive virtual reality (VR) offers the possibility of simulating not only a user's surroundings but also his visual self-representation. In other words, the user sees himself as an avatar rather than seeing his actual body. When the avatar mimics the user's movements with a very short delay, the result is a subjective feeling of ownership of the avatar where the user feels the avatar is his actual physical body. When the virtual body is not aligned with the physical body, users experience a proprioceptive drift where the perceived position of his limbs drifts towards that of the virtual avatar. In recent years, our team has harnessed this illusion through the development of a VR system that has been shown to alter a user's movements through the real-time manipulation of the movements of his avatar. We have found that the user's brain signals can be used to measure the illusion of ownership and that the illusion can be created using vibrators that give the illusion of movement, rather than actual movements.
Our recent progress lead us to build upon this and take our research program to a new level in order to explore and apply the potential of real-time embodied avatars, progressing towards our ambitious long-term goal of developing a complete virtual surrogate body that is controlled and felt by its user. This research program consists of three threads. The 1st aims to integrate and extend our current technologies into a multisensory virtual body controlled by movement or imagined movements and felt through proprioceptive illusions. The 2nd aims to develop a system that allows for gradual real-time deformation of the avatar's upper and lower limbs (unilateral length, girth, segment proportions, etc.) during the embodiment experience in order to modify the user's body self-representation and alter motor control. The 3rd aims to combine a haptic exerciser with our self-avatar. Here the avatar will be driven by the movements and forces produced by users, in order to restore congruency between vision and motor control in post-stroke patients.
This research will lead to fully-embodied avatars that will be used in physical rehabilitation (to study and treat pathologies that cause motor impairment, such as stroke and spinal cord injury), sports (improving sport biomechanics and learning new tasks), telepresence systems (allowing a remote user to feel fully present), robotics (feeling the movements of the robot that is controlled) and full-body surrogacy (allowing paralyzed patients to control a virtual or robotic body). The research program will train 4 graduate students and 4 undergraduates, equipping them with skills that are at the cutting edge of technology and in high demand in Canadian industry. They will learn how to: develop complex multisensory VR systems; develop highly interactive video games and real-time computer animations; acquire, process and interpret biomedical signals; develop novel human-computer interaction (HCI) methods.