Grant: Discovery Grants Program - Individual

If a robot is to successfully navigate from one location to another, it must identify an appropriate sequence of movements to get there. Similarly, a chess-playing program must select a sequence of "good" moves in response to its opponent if the program is to win. These are examples of sequential decision making tasks, which require that an autonomous agent make a sequence of effective decisions about how to act in order to achieve some desired objective. Sequential decision making is a fundamental task in the field of Artificial Intelligence, and it is becoming even more important as autonomous systems become a larger part of daily life. It has been examined by several different research communities, each considering different settings and using different solution methods. This includes reinforcement learning, automated planning, and game-playing systems. The long-term goal of my research agenda is to develop the underlying principles and techniques needed for effective sequential decision making. I will pursue two main lines of inquiry. The first is to improve our understanding of different sequential decision making methods --- specifically reinforcement learning, automated planning, and game-playing approaches --- with the objective of making it easier to develop systems to solve new or more complex sequential decision making tasks. This includes an examination of how different design decisions affect system performance, and an investigation into what properties of a problem lead to one technique being more effective than another. The second main line of research is in understanding how different sequential decision making methods can be used to improve and complement each other. AlphaGo is an excellent example of this idea, as this system used reinforcement learning to generate an evaluation function, and then used this function to guide a planning method during play. This general paradigm is a very powerful one, and better understanding when it is most appropriate or how to best combine other such methods should advance our ability to make systems with strong sequential decision making capabilities. Sequential decision making methods have been used in a variety of applications including DNA sequence alignment, underground sewer placement for new subdivisions, model-based diagnosis of faulty systems, and robotics. However, it is still quite complex to design a system that uses sequential decision making approaches for some new given task, due to the significant number of design decisions that need to be made. Having a better understanding of when different methods are most effective will help speed up this process, while identifying new ways of combining the various methods for sequential decision making will increase the size and complexity of problems that can be solved.