Grant: Discovery Grants Program - Individual
Simon Fraser University
$29,000.00
Efficient Algorithms for Sequential Decision-making
EF 2023-24
Résumé
Machine learning allows computers to automatically detect patterns in data, and leverage it to make predictions or decisions in the real-world. In the last decade, we have witnessed an increasing number of technological and scientific fields gather large amounts of data, and use machine learning techniques for making data-driven decisions. Reinforcement learning (RL) is a subfield of machine learning that focuses on problems that involve making repeated, sequential decisions to interact with the world, collect data and reason about it, all with incomplete information about the world. Applications of such problems include clinical trials in medicine, monitoring industrial plants, robotics, and computational marketing and advertising. Though RL has found great practical success recently, typical practical algorithms are often (i) brittle, meaning that their performance is sensitive to hyper-parameters and minor design decisions, (ii) inefficient in that they require a large number of interactions to learn to make good decisions and, (iii) do not have theoretical guarantees on their performance and can fail on simple, synthetic problems. To alleviate these problems, we propose to develop statistically efficient, computationally tractable algorithms that can easily scale to large sequential decision-making problems. Throughout, our aim will be to develop algorithms that (i) are either parameter-free or robust to hyper-parameter tuning, (ii) can effectively exploit the underlying problem structure and be sample-efficient and (iii) have tight bounds on their worst-case statistical and computational performance for representative problem classes The research program will especially focus on RL problems that need to incorporate constraints while making decisions, trade off multiple conflicting objectives and reason in the presence of other cooperating or competing agents. The proposed research will help bridge the gap between the theory and practice of RL, and also contribute to the adjacent fields of machine learning and numerical optimization. Via collaborations with experts in industry and academia, we aim to use the developed techniques in healthcare, recommendation systems and computational advertising. By making progress towards the research program's objectives, we hope to expand the scope of data-driven decision-making in the real-world. Furthermore, we will provide key training to graduate students equipping them with strong mathematical foundations and programming skills necessary to solve important problems. We expect that the research program will augment Canada's existing strengths in machine learning, and help develop highly-skilled professionals valuable to Canadian institutions and companies.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
British Columbia, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2022-04816
Name nom
Vaswani, Sharan
Department département
Department of Computing Science
Organization id
5
Institution établissement
Simon Fraser University
Province en
British Columbia
Province fr
Colombie-Britannique
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2022
Award amount
29000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
1205
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Mathematical sciences
Area of application fr
Sciences mathématiques
Research subject code
2714
Research subject group en
Information technology
Research subject group fr
Mathematics of computing
Research subject en
Technologie de l'information
Research subject fr
Mathématiques de l'informatique
Application title
Efficient Algorithms for Sequential Decision-making
Keyword
Machine Learning; Optimization; Reinforcement Learning; Sequential decision-making under uncertainty
Application summary
Machine learning allows computers to automatically detect patterns in data, and leverage it to make predictions or decisions in the real-world. In the last decade, we have witnessed an increasing number of technological and scientific fields gather large amounts of data, and use machine learning techniques for making data-driven decisions. Reinforcement learning (RL) is a subfield of machine learning that focuses on problems that involve making repeated, sequential decisions to interact with the world, collect data and reason about it, all with incomplete information about the world. Applications of such problems include clinical trials in medicine, monitoring industrial plants, robotics, and computational marketing and advertising. Though RL has found great practical success recently, typical practical algorithms are often (i) brittle, meaning that their performance is sensitive to hyper-parameters and minor design decisions, (ii) inefficient in that they require a large number of interactions to learn to make good decisions and, (iii) do not have theoretical guarantees on their performance and can fail on simple, synthetic problems. To alleviate these problems, we propose to develop statistically efficient, computationally tractable algorithms that can easily scale to large sequential decision-making problems. Throughout, our aim will be to develop algorithms that (i) are either parameter-free or robust to hyper-parameter tuning, (ii) can effectively exploit the underlying problem structure and be sample-efficient and (iii) have tight bounds on their worst-case statistical and computational performance for representative problem classes The research program will especially focus on RL problems that need to incorporate constraints while making decisions, trade off multiple conflicting objectives and reason in the presence of other cooperating or competing agents. The proposed research will help bridge the gap between the theory and practice of RL, and also contribute to the adjacent fields of machine learning and numerical optimization. Via collaborations with experts in industry and academia, we aim to use the developed techniques in healthcare, recommendation systems and computational advertising. By making progress towards the research program's objectives, we hope to expand the scope of data-driven decision-making in the real-world. Furthermore, we will provide key training to graduate students equipping them with strong mathematical foundations and programming skills necessary to solve important problems. We expect that the research program will augment Canada's existing strengths in machine learning, and help develop highly-skilled professionals valuable to Canadian institutions and companies.