Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Discovery Grants Program - Individual

McGill University

$31,000.00

Variational analysis of optimization problems in machine learning and data science

FY 2024-25

Summary
In recent times, optimization has undergone a renaissance unlike many other disciplines, spurred by the advent of big data and machine learning. These domains have become the `holy grail' for countless researchers in the computational sciences. The optimization problems arising in this era often exhibit nonsmoothness, are rooted in high-dimensional and noisy data, and introduce novel hurdles, sparking innovative research. This rush  of activity comes with an urgent need for foundational mathematical insights to provide theoretical underpinnings and guide progress. At the core of optimization's theoretical foundation lies nonsmooth and set-valued analysis, now commonly referred to as variational analysis. Variational analysis has evolved into its own distinct research area, with vibrant research communities. However, a notable gap persists between the foundational work in variational analysis and the practical tools employed in today's optimization applications. The  long-term objective of this program is to bridge this divide, forging a connection between variational analysis and computational methodologies. Within this program, we concentrate on two main topics, both closely intertwined with modern statistics, that we briefly describe here: 1. Perturbation analysis of nonsmooth optimization problems. We are interested in studying the stability of additive composite optimization problems which model a myriad of optimization problems in data science and learning, such as regularized least-squares (in particular, LASSO and its relatives). Concretely, we would like to establish quantifiable (Lipschitz) bounds for the change of solutions as the problem data changes. Our study will employ and develop tools from modern variational analysis, in particular, second-order differentiation based on graphical differentiation of set-valued maps and the associated set-valued implicit function theorems. 2. The maximum on the mean method for inverse problems. We  study the maximum entropy on the mean (MEM) method for inverse problems, and as a means of statistical estimation. In particular, we want to explore how to incorporate data-driven information about the prior distribution of the data, and of the noise of the data, into the MEM framework. This will take advantage of variational techniques such as epigraphical convergence and convex duality to move seamlessly between dual, primal and approximate problems based on empirical estimations. Outcomes and Significance We anticipate in 1. to obtain concrete Lipschitz bounds on the solution functions studied that will yield inside into parameter-tuning. Moreover, the anticipated smoothness (e.g., semismoothness) properties will allow us to use the solution maps in concrete algorithms. In 2. we anticipate to arrive at a data-driven, mathematically well-founded algorithmic toolbox that practitioners can use to solve inverse problems from various areas, e.g., image processing and compressed sensing.
Program
Discovery Grants Program - Individual
Location
Quebec, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
RGPIN-2024-04116
Name nom
Hoheisel, Tim
Department département
Mathematics and Statistics
Organization id
61
Institution établissement
McGill University
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2024
Award amount
31000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1508
Committee name en
Mathematics and Statistics
Committee name fr
Mathématiques et statistique
Area of application code
1205
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Mathematical sciences
Area of application fr
Sciences mathématiques
Research subject code
2956
Research subject group en
Applied mathematics
Research subject group fr
Mathématiques appliquées
Research subject en
Optimisation and optimal control theory
Research subject fr
Optimisation et théories de commande optimale
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Variational analysis of optimization problems in machine learning and data science
Keywords
variational analysis; graphical differentiation; convex duality; maximum entropy on the mean; inverse problem; stability analysis; epigraphical convergence; subdifferential; nonsmooth optimization; convex optimization
Application summary
In recent times, optimization has undergone a renaissance unlike many other disciplines, spurred by the advent of big data and machine learning. These domains have become the `holy grail' for countless researchers in the computational sciences. The optimization problems arising in this era often exhibit nonsmoothness, are rooted in high-dimensional and noisy data, and introduce novel hurdles, sparking innovative research. This rush  of activity comes with an urgent need for foundational mathematical insights to provide theoretical underpinnings and guide progress. At the core of optimization's theoretical foundation lies nonsmooth and set-valued analysis, now commonly referred to as variational analysis. Variational analysis has evolved into its own distinct research area, with vibrant research communities. However, a notable gap persists between the foundational work in variational analysis and the practical tools employed in today's optimization applications. The  long-term objective of this program is to bridge this divide, forging a connection between variational analysis and computational methodologies. Within this program, we concentrate on two main topics, both closely intertwined with modern statistics, that we briefly describe here: 1. Perturbation analysis of nonsmooth optimization problems. We are interested in studying the stability of additive composite optimization problems which model a myriad of optimization problems in data science and learning, such as regularized least-squares (in particular, LASSO and its relatives). Concretely, we would like to establish quantifiable (Lipschitz) bounds for the change of solutions as the problem data changes. Our study will employ and develop tools from modern variational analysis, in particular, second-order differentiation based on graphical differentiation of set-valued maps and the associated set-valued implicit function theorems. 2. The maximum on the mean method for inverse problems. We  study the maximum entropy on the mean (MEM) method for inverse problems, and as a means of statistical estimation. In particular, we want to explore how to incorporate data-driven information about the prior distribution of the data, and of the noise of the data, into the MEM framework. This will take advantage of variational techniques such as epigraphical convergence and convex duality to move seamlessly between dual, primal and approximate problems based on empirical estimations. Outcomes and Significance We anticipate in 1. to obtain concrete Lipschitz bounds on the solution functions studied that will yield inside into parameter-tuning. Moreover, the anticipated smoothness (e.g., semismoothness) properties will allow us to use the solution maps in concrete algorithms. In 2. we anticipate to arrive at a data-driven, mathematically well-founded algorithmic toolbox that practitioners can use to solve inverse problems from various areas, e.g., image processing and compressed sensing.