Grant: Discovery Grants Program - Individual
University of Waterloo
$33,000.00
Splitting methods in optimization: Beyond consistency and convexity.
EF 2023-24
Résumé
Optimization problems, which appear in many fields of science, engineering and healthcare, ask to find minimizers or maximizers of functions subject to constraints. Splitting algorithms are powerful optimization methods which aim to solve constrained convex (but not necessarily smooth) optimization problems, which are oftentimes encountered in practice. Examples of these methods are the Douglas-Rachford method, the forward-backward method and the alternating direction method of multipliers (ADMM). Concrete areas of applications include image processing (e.g., image denoising problems), data science (e.g., support vector machine problems), statistics and machine learning (e.g., the least absolute shrinkage and selection operator (LASSO) problem) and finance (e.g., portfolio optimization). Finding conditions that guarantee the convergence of these algorithms to a solution is a very active area of research. Typical convergence results require the convexity of the objective function and/or the constraints, as well as the existence of solutions. The long-term objective of my research program is to investigate the behaviour of splitting methods in the absence of minimizers and/or convexity. The importance of investigating this situation stems from the fact that it occurs in practice but is not well understood. These aspects of my research connect to phase retrieval, intensity-modulated radiation therapy, and deep learning (via the popular but nonconvex loss function from neural networks). The short-term objectives of my research program are summarized below. 1. Using tools from convex and variational analysis, I will explore the analytic behaviour of different splitting algorithms in the inconsistent and/or nonconvex case. Some of these methods have shown great promise when applied to these cases; however, an in-depth analysis is still lacking. 2. Building on these analytical results, I will utilize techniques from dynamical systems and nonlinear operator theory to develop the algorithmic counterpart of the analysis. In this case, we must deal with operators (for instance, projections onto nonconvex sets) that lack properties typically utilized in convergence proofs. 3. The third component of my research program is the computational counterpart. This research includes numerical testing and comparison of the computational cost of these methods to other existing ones, as well as developing software packages. Impact: On the one hand, the progress we make will be of significant interest for scientists employing splitting methods as well as the industrial and healthcare sectors, since the functions used to model industrial problems are often nonconvex. On the other hand, my students and postdoctoral fellows will acquire a strong analytic and computational foundation. This will contribute to developments in optimization theory and related software. Both aspects will significantly contribute to Canada's leading role in science and technology.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Ontario, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2019-04803
Name nom
Moursi, Walaa
Department département
Combinatorics and Optimization
Organization id
33
Institution établissement
University of Waterloo
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2019
Award amount
33000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
1205
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Mathematical sciences
Area of application fr
Sciences mathématiques
Research subject code
1601
Research subject group en
Industrial engineering
Research subject group fr
Operations research and management science
Research subject en
Génie industriel
Research subject fr
Recherche opérationnelle et science de la gestion
Application title
Splitting methods in optimization: Beyond consistency and convexity.
Keyword
convex analysis; convex optimization; Douglas-Rachford method; forward-backward method; monotone operator; optimization algorithms; projection operator; proximal mapping; splitting method; variational analysis
Application summary
Optimization problems, which appear in many fields of science, engineering and healthcare, ask to find minimizers or maximizers of functions subject to constraints. Splitting algorithms are powerful optimization methods which aim to solve constrained convex (but not necessarily smooth) optimization problems, which are oftentimes encountered in practice. Examples of these methods are the Douglas-Rachford method, the forward-backward method and the alternating direction method of multipliers (ADMM). Concrete areas of applications include image processing (e.g., image denoising problems), data science (e.g., support vector machine problems), statistics and machine learning (e.g., the least absolute shrinkage and selection operator (LASSO) problem) and finance (e.g., portfolio optimization). Finding conditions that guarantee the convergence of these algorithms to a solution is a very active area of research. Typical convergence results require the convexity of the objective function and/or the constraints, as well as the existence of solutions. The long-term objective of my research program is to investigate the behaviour of splitting methods in the absence of minimizers and/or convexity. The importance of investigating this situation stems from the fact that it occurs in practice but is not well understood. These aspects of my research connect to phase retrieval, intensity-modulated radiation therapy, and deep learning (via the popular but nonconvex loss function from neural networks). The short-term objectives of my research program are summarized below. 1. Using tools from convex and variational analysis, I will explore the analytic behaviour of different splitting algorithms in the inconsistent and/or nonconvex case. Some of these methods have shown great promise when applied to these cases; however, an in-depth analysis is still lacking. 2. Building on these analytical results, I will utilize techniques from dynamical systems and nonlinear operator theory to develop the algorithmic counterpart of the analysis. In this case, we must deal with operators (for instance, projections onto nonconvex sets) that lack properties typically utilized in convergence proofs. 3. The third component of my research program is the computational counterpart. This research includes numerical testing and comparison of the computational cost of these methods to other existing ones, as well as developing software packages. Impact: On the one hand, the progress we make will be of significant interest for scientists employing splitting methods as well as the industrial and healthcare sectors, since the functions used to model industrial problems are often nonconvex. On the other hand, my students and postdoctoral fellows will acquire a strong analytic and computational foundation. This will contribute to developments in optimization theory and related software. Both aspects will significantly contribute to Canada's leading role in science and technology.