Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Engage Grants Program

University of Calgary

$25,000.00

Immersive Visualization of Reinforcement Learning

FY 2018-19

Summary
Borealis AI is the Royal Bank of Canada (RBC) Institute for Research, pursuing fundamental and applied research in reinforcement learning (RL), deep learning, adversarial machine learning, and other related areas. In addition to applying state of the art methods, they develop new RL techniques specifically designed to address these real-world challenges. One of the fundamental improvements to the framework is by incorporating human knowledge - leveraging a user's abilities can help the system learn faster and better across a wide range of tasks. The long-term goal of this research is to learn from both explicit and implicit user feedback to the system. In order to make progress towards this goal, there are two challenges that must be overcome. First, the non-technical trainer must be able to understand an agent's policy, or way of acting. We hypothesize that the better the trainer understands what the agent does and does not know, the better she will be able to provide assistance. Second, the Borealis AI RL experts must better understand how a trainer's inputs are affecting the algorithm. We hypothesize that moving from the standard black box paradigm to one in which learning is more transparent will allow us to develop new and improved RL methods to better leverage human feedback, as well as to help set the numerous hyper-parameters involved in learning. Borealis AI is looking for help to apply visualization techniques to attack these two critical problems. *The project's goal is to investigate whether immersive visualizations can improve understanding of the RL process for both non-technical and expert users. We will explore visualization techniques in Virtual and Augmented Reality (VR and AR) using the HTC Vive and the Microsoft HoloLens to answer the following questions: 1) What are different ways to visualize the policies that agents use for learning in an immersive environment, which helps a lay audience understand the policies? 2) How can we visualize the algorithm's internal data (e.g., a neural network) in an immersive environment to demonstrate how the knowledge is evolving during the learning process?*
Program
Engage Grants Program
Location
Alberta, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
531581-2018
Name nom
Maurer, Frank
Department département
Computer Science
Organization id
11
Institution établissement
University of Calgary
Province en
Alberta
Province fr
Alberta
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2018
Competition year année de concours
2018
Award amount
25000
Program id
EGP
Program name en
Engage Grants Program
Program name fr
Programme de subventions d'engagement partenariat
Group en
Research partnerships programs
Group fr
Programmes de partenariat
Committee code
1555
Committee name en
Prairie Internal Decision Committee
Committee name fr
Comité de décision interne pour les Prairies
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
2716
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Virtual reality and related simulations
Research subject fr
Réalité virtuelle et simulations connexes
Partie
1
Nb partie
1
Application title
Immersive Visualization of Reinforcement Learning
Keyword
NSERC
Application summary
Borealis AI is the Royal Bank of Canada (RBC) Institute for Research, pursuing fundamental and applied research in reinforcement learning (RL), deep learning, adversarial machine learning, and other related areas. In addition to applying state of the art methods, they develop new RL techniques specifically designed to address these real-world challenges. One of the fundamental improvements to the framework is by incorporating human knowledge - leveraging a user's abilities can help the system learn faster and better across a wide range of tasks. The long-term goal of this research is to learn from both explicit and implicit user feedback to the system. In order to make progress towards this goal, there are two challenges that must be overcome. First, the non-technical trainer must be able to understand an agent's policy, or way of acting. We hypothesize that the better the trainer understands what the agent does and does not know, the better she will be able to provide assistance. Second, the Borealis AI RL experts must better understand how a trainer's inputs are affecting the algorithm. We hypothesize that moving from the standard black box paradigm to one in which learning is more transparent will allow us to develop new and improved RL methods to better leverage human feedback, as well as to help set the numerous hyper-parameters involved in learning. Borealis AI is looking for help to apply visualization techniques to attack these two critical problems. *The project's goal is to investigate whether immersive visualizations can improve understanding of the RL process for both non-technical and expert users. We will explore visualization techniques in Virtual and Augmented Reality (VR and AR) using the HTC Vive and the Microsoft HoloLens to answer the following questions: 1) What are different ways to visualize the policies that agents use for learning in an immersive environment, which helps a lay audience understand the policies? 2) How can we visualize the algorithm's internal data (e.g., a neural network) in an immersive environment to demonstrate how the knowledge is evolving during the learning process?*