Grant: Discovery Grants Program - Individual

Many practical design and operational problems involve sequential decision-making over a planning horizon, where decisions are impacted by uncertain parameters revealed gradually over time. Such problems arise in supply chain resource planning, healthcare staffing and scheduling, telecommunications provisioning, and more. Despite their ubiquity, they remain theoretically and computationally challenging. This research aims to advance two subfields of sequential decision-making under uncertainty: (i) multistage stochastic programming (MSP) to optimize a measure over a finite horizon given a probability distribution, and (ii) multistage adaptive robust optimization (MSARO) to protect against worst-case outcomes when only the possible realizations of uncertainty are known. The proposed research program will focus on imposing structure on MSP and MSARO solutions. Building on recent advancements in novel decision rules, it will focus on developing new methods for under-explored classes of MSP and MSARO, without requiring strong assumptions. It will also extend sequential decision-making to new applications lacking viable solution methodologies. The first research direction of this program will be the design of nonlinear policies for MSP and MSARO problems. Linear decision rules enable computational efficiency but sacrifice optimality. Even in well-studied two-stage programs, optimal policies are piecewise linear. Nonlinear rules could better approximate high-quality policies, but approximating MSP/MSARO with them is notoriously difficult, especially with integer decisions. This research will derive informed policies by incorporating nonlinear rules into reformulations and relaxations of mixed-integer MSP/MSARO. The proposed techniques will be used to model and solve challenging problems in telecommunications, healthcare, and disaster management. In its second research direction, this program will explore the integration of deep reinforcement learning (RL) into the design of solution methodologies for MSP and MSARO. Continuing the discussion of decision rule design above, one research topic will focus on the development of neural networks that serve as decision rules. In this case, decisions in a model will not be limited to predefined functions, but rather neural networks trained on scenarios. Another approach is development of decomposition methods using RL agents to solve subproblems at each stage. A key challenge is designing optimality cuts to guarantee convergence, as ideas from two-stage stochastic programming do not directly extend to multistage problems. Sequential decision-making under uncertainty is crucial for businesses, governments, and individuals, so progress in this area can yield significant cost savings and service improvements. Through methodological contributions to MSP and MSARO and expanding the solvability of their applications, this research will profoundly impact decision-making in engineering and management.