Grant: Discovery Grants Program - Individual
McGill University
$46,000.00
Decentralized stochastic control of multi-agent teams: approximation, learning, and signaling
EF 2023-24
Résumé
We are moving towards an envisioned future where multiple interconnected autonomous agents will interact with humans in shared environments. Examples include self-driving cars, robotic assistants in homes, factory floors and warehouses, Industry 4.0 where automated control algorithms supervised by human operators control multiple interconnected industrial plants, and so on. A salient feature of such environments is that the agents have different information, yet they need to cooperate and coordinate their actions to achieve a common goal. The agents may have uncertainty about the system model and must be able to adapt to stochastic changes in the environment. The long term goal of the proposed research program is to develop theory and algorithms which address these salient features, provide a systematic methodology to design multiple agents operating in dynamic, stochastic, and uncertain environments and, thereby, enable the technologies of the future. The proposal maps a five year research program to pursue three research directions: (i) Approximation guarantees in decentralized control: Quantify the affect of model uncertainty and model approximation on the performance of decentralized systems. Use these to develop a solution framework which provides approximately optimal policy for hitherto unsolved information structures and apply it to networked control systems (ii) Learning with decentralized information: Develop multi-agent reinforcement learning (MARL) framework for decentralized learning and decentralized execution paradigm. Characterize the asymptotic optimality and regret of MARL algorithms. Identify trade-offs between speed of convergence and performance of the converged policies by restricting attention to policies with specific structure. Use this trade-off to investigate explainable and interpretable decision making in human-robot teams. (iii) Role of signaling in multi-agent systems: Characterize what and when to communicate over explicit communication channels when communication is costly and potentially the system model is unknown. Build on these results to characterize how to when and how to signal information via implicit communication. Determine the impact of implicit communication on explainable and interpretable decision making in human-robot teams. The proposed research program will provide a broad training to 5 PhD, 3 MEng, and 5 UG students in fundamental areas of Systems and Control and Reinforcement Learning, thereby providing them with a solid foundation to be at the forefront of innovation of a growing and transformative research field. The results will advance the state of knowledge in decentralized stochastic control and multi-agent reinforcement learning, and will contribute to the emergence of new technologies which will maintain Canada's position as an innovator in machine learning, energy, automotive, aerospace, and information technology sectors.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Quebec, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2021-03511
Name nom
Mahajan, Aditya
Department département
Electrical and Computer Engineering - Electrical and Computer Engineering
Organization id
61
Institution établissement
McGill University
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2021
Award amount
46000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1510
Committee name en
Electrical and Computer Engineering
Committee name fr
Génie électrique et informatique
Area of application code
1200
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Advancement of knowledge
Area of application fr
Avancement des connaissances
Research subject code
9999
Research subject group en
Not available
Research subject group fr
Not available
Research subject en
Non disponible
Research subject fr
Non disponible
Application title
Decentralized stochastic control of multi-agent teams: approximation, learning, and signaling
Keyword
Decentralized stochastic control; Explainable decision making; Human-robot teams; Information structures; Multi-agent reinforcement learning; Multi-agent systems; Reinforcement learning; Team Theory
Application summary
We are moving towards an envisioned future where multiple interconnected autonomous agents will interact with humans in shared environments. Examples include self-driving cars, robotic assistants in homes, factory floors and warehouses, Industry 4.0 where automated control algorithms supervised by human operators control multiple interconnected industrial plants, and so on. A salient feature of such environments is that the agents have different information, yet they need to cooperate and coordinate their actions to achieve a common goal. The agents may have uncertainty about the system model and must be able to adapt to stochastic changes in the environment. The long term goal of the proposed research program is to develop theory and algorithms which address these salient features, provide a systematic methodology to design multiple agents operating in dynamic, stochastic, and uncertain environments and, thereby, enable the technologies of the future. The proposal maps a five year research program to pursue three research directions: (i) Approximation guarantees in decentralized control: Quantify the affect of model uncertainty and model approximation on the performance of decentralized systems. Use these to develop a solution framework which provides approximately optimal policy for hitherto unsolved information structures and apply it to networked control systems (ii) Learning with decentralized information: Develop multi-agent reinforcement learning (MARL) framework for decentralized learning and decentralized execution paradigm. Characterize the asymptotic optimality and regret of MARL algorithms. Identify trade-offs between speed of convergence and performance of the converged policies by restricting attention to policies with specific structure. Use this trade-off to investigate explainable and interpretable decision making in human-robot teams. (iii) Role of signaling in multi-agent systems: Characterize what and when to communicate over explicit communication channels when communication is costly and potentially the system model is unknown. Build on these results to characterize how to when and how to signal information via implicit communication. Determine the impact of implicit communication on explainable and interpretable decision making in human-robot teams. The proposed research program will provide a broad training to 5 PhD, 3 MEng, and 5 UG students in fundamental areas of Systems and Control and Reinforcement Learning, thereby providing them with a solid foundation to be at the forefront of innovation of a growing and transformative research field. The results will advance the state of knowledge in decentralized stochastic control and multi-agent reinforcement learning, and will contribute to the emergence of new technologies which will maintain Canada's position as an innovator in machine learning, energy, automotive, aerospace, and information technology sectors.