Grant: Discovery Grants Program - Individual

Stochastic control studies the control and optimization of dynamical systems in the presence of uncertainty under a variety of models and information structures, with many applications in modern engineering systems. This proposal is on a coherent research program on stochastic control for complex systems in view of partial information and decentralization; approximations and robustness; reinforcement learning for general spaces; and interaction of control and information theories. Robustness to Incorrect Models and Data-Driven Control Often an engineer is given an approximate nominal model. The application of the nominally designed policy to a true model may lead to a loss that may be large. Our earlier results rigorously showed the dependence of robustness on model regularity. We will generalize these results for general information structures with partial and decentralized information, continuous-time processes, and empirical learning including sample complexity. We will also study robustness to independent noise idealizations and Brownian noise idealizations. Stochastic Control with Complex Environments under Partial and Decentralized Information For a partially observable problem, where a controller does not have access to the state, there exist few rigorous results on approximations. We recently showed that under filter stability finite memory policies are near optimal. We will study conditions for controlled filter stability and robustness to incorrect priors. Another informational constraint is due to decentralization when there are multiple agents who have access only to local information. We will develop rigorous discrete-time approximations of continuous-time models with general information. Reinforcement Learning for Complex Systems Decision makers may not have a true model and may need to rely on data to obtain solutions. We have made contributions to reinforcement learning for fully observed continuous-space models, partially observed models and multi-agent models. We will develop a non-Markovian learning framework for general information structures and various criteria. For complex multi-agent systems, we will study equilibria which allow for computational execution and convergence. Information Theory and Networked Control Stochastic networked control refers to decentralized systems with communication channels. Building on our prior work on networked control and zero-delay coding, we will develop reinforcement learning algorithms for optimal policies. We will develop coding theorems for unstable systems via separating stability and optimality. Finally, we will continue working on viewing coding as a decentralized optimization over the space of information kernels, especially for partially observable and multi-agent models. The theoretically foundational and practically highly consequential research, and the HQP training consisting of 6 PhD, 4 MSc, 2 Postdocs and 4 USRA will contribute to Canada's leadership in innovation.