Grant: Discovery Grants Program - Individual
University of Toronto
$50,000.00
Risk aware stochastic control and games
EF 2024-25
Résumé
Risk is all around us, and for our decisions under uncertainty to be sound, we need to properly account for risk when weighing strategies. In the proposal, I will highlight two examples that are important to society that pose significant scientific challenges. First, when sequestering carbon by injecting CO2 into porous rock, naively injecting at the highest rate (storing most CO2) can induce seismic activity due to subsequent pressure buildup with porosity and permeability uncertainty introducing (latent) risk factors. Second, when controlling robots to explore unknown territories with limited fuel, there is the risk of taking fuel-efficient trajectories that place the robot in danger (e.g., following along a cliff too closely).
The main objective of this research program is to develop the theory and methodology for solving optimization problems that utilize risk aware criteria and are time-consistent in both single- and multi-agent settings. In tandem, I propose to develop reinforcement learning (RL) algorithms to provide provably approximate solutions to such problems.
In stochastic control and games, traditionally, agents evaluate a particular strategy by using expected (discounted) running costs plus terminal costs as criteria. Such criteria tacitly do not account for risk. Two approaches are often taken to address this limitation: i) naively adding fictitious costs, e.g., the probability of a state variable being below some level (i.e., an underperformance cost), and ii) using expected utility criteria. Economic theory, however, tells us that these alone cannot account for the full spectrum of agents' risk preferences.
Instead, my research program proposes to incorporate time-consistent dynamic risk measures to assess the risk in agent's strategies and is designed in two research themes: 1) incorporating partial information and, in the multi-agent setting, differing beliefs and preferences into decision making, and 2) developing RL methodology that can approximate solutions in a data driven manner. The existing theory devoted to risk measures are often focused on their mathematical properties, characterizations, and dual representations, while only a few focus on their optimization or on multi-agent settings.
The outcomes of this research will be new theory and methodology (in the form of journal articles) and public code repositories that have wide applicability, including: optimal carbon capture and sequestration, risks in decentralized finance, and optimal contracting for regulated firms in offset credit markets.
The results of this research will benefit Canada as it will assist regulators to make better decisions on market design and to better understand how risk-averse agents may react to various environments. It will also ensure Canadian researchers are at the leading edge of research in important aspects of the data science/artificial intelligent landscape and train HQP who are sought after by industry and academia.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Ontario, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2024-04317
Name nom
Jaimungal, Sebastian
Department département
Statistical Sciences
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2024
Award amount
50000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1508
Committee name en
Mathematics and Statistics
Committee name fr
Mathématiques et statistique
Area of application code
1205
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Mathematical sciences
Area of application fr
Sciences mathématiques
Research subject code
3008
Research subject group en
Statistics and probability
Research subject group fr
Statistique et probabilité
Research subject en
Applied probability
Research subject fr
Probabilité appliquée
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Risk aware stochastic control and games
Keywords
dynamic risk measures; risk aware reinforcement learning; mean field games; princpal agent problems
Application summary
Risk is all around us, and for our decisions under uncertainty to be sound, we need to properly account for risk when weighing strategies. In the proposal, I will highlight two examples that are important to society that pose significant scientific challenges. First, when sequestering carbon by injecting CO2 into porous rock, naively injecting at the highest rate (storing most CO2) can induce seismic activity due to subsequent pressure buildup with porosity and permeability uncertainty introducing (latent) risk factors. Second, when controlling robots to explore unknown territories with limited fuel, there is the risk of taking fuel-efficient trajectories that place the robot in danger (e.g., following along a cliff too closely).
The main objective of this research program is to develop the theory and methodology for solving optimization problems that utilize risk aware criteria and are time-consistent in both single- and multi-agent settings. In tandem, I propose to develop reinforcement learning (RL) algorithms to provide provably approximate solutions to such problems.
In stochastic control and games, traditionally, agents evaluate a particular strategy by using expected (discounted) running costs plus terminal costs as criteria. Such criteria tacitly do not account for risk. Two approaches are often taken to address this limitation: i) naively adding fictitious costs, e.g., the probability of a state variable being below some level (i.e., an underperformance cost), and ii) using expected utility criteria. Economic theory, however, tells us that these alone cannot account for the full spectrum of agents' risk preferences.
Instead, my research program proposes to incorporate time-consistent dynamic risk measures to assess the risk in agent's strategies and is designed in two research themes: 1) incorporating partial information and, in the multi-agent setting, differing beliefs and preferences into decision making, and 2) developing RL methodology that can approximate solutions in a data driven manner. The existing theory devoted to risk measures are often focused on their mathematical properties, characterizations, and dual representations, while only a few focus on their optimization or on multi-agent settings.
The outcomes of this research will be new theory and methodology (in the form of journal articles) and public code repositories that have wide applicability, including: optimal carbon capture and sequestration, risks in decentralized finance, and optimal contracting for regulated firms in offset credit markets.
The results of this research will benefit Canada as it will assist regulators to make better decisions on market design and to better understand how risk-averse agents may react to various environments. It will also ensure Canadian researchers are at the leading edge of research in important aspects of the data science/artificial intelligent landscape and train HQP who are sought after by industry and academia.