Grant: Discovery Grants Program - Individual
The University of British Columbia
$44,000.00
Fundamentals of Modern Machine Learning: A Precise High-dimensional Approach
EF 2024-25
Résumé
As we aspire to use data-driven machine-learning (ML) algorithms to create automated decision rules in more aspects of everyday life, we need to make sure that they meet a number of complex system requirements: ML algorithms used for perception in self-driving cars need to be safe against disturbances caused by adversaries. In applications that directly involve data about people, such as decisions on who is granted a loan or who gets hired, we need to ensure fairness against demographic imbalances that exist in our society and translate to data. To effectively use modern deep-learning models -which are increasingly more complex, thus computationally expensive- in resource constrained platforms such as mobile health devices, we need to carefully balance accuracy and resource efficiency. The goal of my research program is to advance the expanded use of ML by developing a modern theory that can guide the design of algorithms that fulfill these requirements.
A prime challenge in developing such a theory is the high-dimensionality of data that renders classical statistical tools inadequate. But even where recent theories have captured certain aspects of high-dimensionality, they have often failed to capture newly discovered ML phenomena, because they produce statistical characterizations that are not precise.
To address these challenges, I will develop a new `precise high-dimensional (HD) statistics' approach to modern ML theory. I will establish a mathematical framework that will lead to precise characterization of the accuracy of classification algorithms as a function of the distribution and size of data, the model complexity, and the algorithms' parameters. This effort builds on my previous work, which innovated a method of precise estimation-error analysis in HD signal processing. Now I will apply the new framework to guide the design of improved ML algorithms with three objectives in mind: robustness to adversarial perturbations (aka safety), robustness to imbalances (aka fairness) and reduced model complexity (aka resource efficiency). To this end, I will also develop theory-driven statistical models that are rich enough to resemble the intricacies of data-driven ones.
This program will provide students with the essential tools in mathematical data science: optimization, probability, statistical signal-processing, and learning theories. Just as awareness of biases in data and algorithms are key concerns of my research, I am also committed to building a diverse research group through inclusive recruitment, training environment, and teaching.
The focus of my research program aligns with Canada's national strategy for AI with a particular emphasis on robustness and equitable algorithms for protecting the rights of marginalized groups. The outcomes of the proposed program have the potential to lead to tech-industry collaborations towards integrating the new provably robust and resource-efficient algorithms to existing data-driven products.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
British Columbia, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2021-03677
Name nom
Thrampoulidis, Christos
Department département
ECE
Organization id
2
Institution établissement
The University of British Columbia
Province en
British Columbia
Province fr
Colombie-Britannique
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2021
Award amount
44000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1510
Committee name en
Electrical and Computer Engineering
Committee name fr
Génie électrique et informatique
Area of application code
1200
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Advancement of knowledge
Area of application fr
Avancement des connaissances
Research subject code
2805
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Learning and inference theories
Research subject fr
Théories de l'apprentissage et l'inférence
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Fundamentals of Modern Machine Learning: A Precise High-dimensional Approach
Keywords
fundamental limits; sharp asymptotics; high-dimensional data; gaussian process inequalities; multiclass classification; adversarial training; robust learning; overparameterization; equitable learning; model pruning/compression
Application summary
As we aspire to use data-driven machine-learning (ML) algorithms to create automated decision rules in more aspects of everyday life, we need to make sure that they meet a number of complex system requirements: ML algorithms used for perception in self-driving cars need to be safe against disturbances caused by adversaries. In applications that directly involve data about people, such as decisions on who is granted a loan or who gets hired, we need to ensure fairness against demographic imbalances that exist in our society and translate to data. To effectively use modern deep-learning models -which are increasingly more complex, thus computationally expensive- in resource constrained platforms such as mobile health devices, we need to carefully balance accuracy and resource efficiency. The goal of my research program is to advance the expanded use of ML by developing a modern theory that can guide the design of algorithms that fulfill these requirements.
A prime challenge in developing such a theory is the high-dimensionality of data that renders classical statistical tools inadequate. But even where recent theories have captured certain aspects of high-dimensionality, they have often failed to capture newly discovered ML phenomena, because they produce statistical characterizations that are not precise.
To address these challenges, I will develop a new `precise high-dimensional (HD) statistics' approach to modern ML theory. I will establish a mathematical framework that will lead to precise characterization of the accuracy of classification algorithms as a function of the distribution and size of data, the model complexity, and the algorithms' parameters. This effort builds on my previous work, which innovated a method of precise estimation-error analysis in HD signal processing. Now I will apply the new framework to guide the design of improved ML algorithms with three objectives in mind: robustness to adversarial perturbations (aka safety), robustness to imbalances (aka fairness) and reduced model complexity (aka resource efficiency). To this end, I will also develop theory-driven statistical models that are rich enough to resemble the intricacies of data-driven ones.
This program will provide students with the essential tools in mathematical data science: optimization, probability, statistical signal-processing, and learning theories. Just as awareness of biases in data and algorithms are key concerns of my research, I am also committed to building a diverse research group through inclusive recruitment, training environment, and teaching.
The focus of my research program aligns with Canada's national strategy for AI with a particular emphasis on robustness and equitable algorithms for protecting the rights of marginalized groups. The outcomes of the proposed program have the potential to lead to tech-industry collaborations towards integrating the new provably robust and resource-efficient algorithms to existing data-driven products.