Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Discovery Grants Program - Individual

University of Waterloo

$41,000.00

Machine Learning Beyond Prediction - Extracting Insights and Guiding Actions

FY 2021-22

Summary
Owing to breakthroughs in supervised learning using deep neural networks, applications of machine learning (ML) have proliferated, spreading to countless industries and societal decisions. Amid this excitement, some fundamental obstacles are often ignored. While ML systems are typically trained to estimate conditional probabilities, in automated systems their primary purpose is to guide actions.  The discrepancy between predictions and decisions is just one among many mismatches between the supervised learning formalism and real-world goals. For example, often the purpose of training a model is not simply to make a prediction but rather to extract qualitative insights, such as causal inference, data clustering or outlier detection.However, when machine learning tools are applied to extract any of those insights, strong assumptions (such as the data being generated by some parameterized probability distribution) are used, often implicitly. Just the same, machine learning is applied far beyond the strict confines of those assumptions. On the other end, for deriving hardness results, most of the theoretical analysis of required resources (be it computational time or training sample sizes) refer to worst-case scenarios, therefore being overly pessimistic. Under that view, large neural networks seem doomed to fail. This project addresses three areas of discrepancy between ML formalism and such real world goals. 1)Interpretability of ML-based tools: Traditional measures of success, such as statistical accuracy and computational efficiency, do not suffice for human consequential applications, where society expects accountability and interpretability. I will analyze formal notions of interpretability and investigate how such notions effect prediction accuracy and render models amenable to monitoring. I will develop theoretical principles under which today's deep learning tools can be leveraged to confer insights beyond their predictive accuracy. 2) Guided selection of clustering algorithms: In spite of the major practical importance of unsupervised learning, current practical implementations of such tasks are very rudimentary. There exists no methodical guidance for clustering tool selection for a given clustering task. I shall address this crucial lacuna by developing methods to guide task appropriate choices of clustering paradigms. 3) Alternatives to worst-case analysis of ML tasks: Many optimization problems that arise in machine learning are NP hard. For example, the training of even small neural networks. Just the same, such problems are being handled routinely on real data for many applications. Experimental evidence suggests that this success relies on some "tameness" of practically arising data. We propose to address this theory-practice discrepancy by distilling structural properties of inputs that can be assumed to hold for naturally arising input data, while giving rise to efficient algorithms for solving hard problems on such inputs.
Program
Discovery Grants Program - Individual
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
RGPIN-2020-04333
Name nom
Ben-David, Shai
Department département
Computer Science, David R. Cheriton School of - Computer Science, David R. Cheriton School of
Organization id
33
Institution établissement
University of Waterloo
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2021
Competition year année de concours
2020
Award amount
41000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
1207
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Information, computer and communication technologies
Area of application fr
Technologie de l'information, des ordinateurs et des réseaux de communications
Research subject code
2805
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Learning and inference theories
Research subject fr
Théories de l'apprentissage et l'inférence
Installment
inst:2
Partie
1
Nb partie
5
Application title
Machine Learning Beyond Prediction - Extracting Insights and Guiding Actions
Keyword
NSERC
Application summary
Owing to breakthroughs in supervised learning using deep neural networks, applications of machine learning (ML) have proliferated, spreading to countless industries and societal decisions. Amid this excitement, some fundamental obstacles are often ignored. While ML systems are typically trained to estimate conditional probabilities, in automated systems their primary purpose is to guide actions.  The discrepancy between predictions and decisions is just one among many mismatches between the supervised learning formalism and real-world goals. For example, often the purpose of training a model is not simply to make a prediction but rather to extract qualitative insights, such as causal inference, data clustering or outlier detection.However, when machine learning tools are applied to extract any of those insights, strong assumptions (such as the data being generated by some parameterized probability distribution) are used, often implicitly. Just the same, machine learning is applied far beyond the strict confines of those assumptions. On the other end, for deriving hardness results, most of the theoretical analysis of required resources (be it computational time or training sample sizes) refer to worst-case scenarios, therefore being overly pessimistic. Under that view, large neural networks seem doomed to fail. This project addresses three areas of discrepancy between ML formalism and such real world goals. 1)Interpretability of ML-based tools: Traditional measures of success, such as statistical accuracy and computational efficiency, do not suffice for human consequential applications, where society expects accountability and interpretability. I will analyze formal notions of interpretability and investigate how such notions effect prediction accuracy and render models amenable to monitoring. I will develop theoretical principles under which today's deep learning tools can be leveraged to confer insights beyond their predictive accuracy. 2) Guided selection of clustering algorithms: In spite of the major practical importance of unsupervised learning, current practical implementations of such tasks are very rudimentary. There exists no methodical guidance for clustering tool selection for a given clustering task. I shall address this crucial lacuna by developing methods to guide task appropriate choices of clustering paradigms. 3) Alternatives to worst-case analysis of ML tasks: Many optimization problems that arise in machine learning are NP hard. For example, the training of even small neural networks. Just the same, such problems are being handled routinely on real data for many applications. Experimental evidence suggests that this success relies on some "tameness" of practically arising data. We propose to address this theory-practice discrepancy by distilling structural properties of inputs that can be assumed to hold for naturally arising input data, while giving rise to efficient algorithms for solving hard problems on such inputs.