Grant: Discovery Grants Program - Individual
University of Toronto
$29,000.00
Trustworthy Machine Learning by Design
EF 2021-22
Résumé
Machine learning (ML) brought a new computing paradigm: rather than writing programs for each problem we'd like to solve, we now write a single learning algorithm and present it with examples of inputs and outputs that are expected from the solution to the problem. This has fundamentally changed the way we build software and enabled progress in many previously challenging areas such as autonomous driving, healthcare, or computer vision. Yet, the widespread adoption of ML raises security and societal concerns that point to the lack of trustworthiness of algorithms used for training and predicting. Learning algorithms can easily be manipulated by adversaries capable of perturbing the data that ML algorithms analyze. Learning algorithms can also perpetuate and magnify biases historically found in the data they analyze. These challenges remain unaddressed despite significant interest from the research community, because it is significantly more difficult to ensure that the logic followed to solve a problem satisfies desiderata (such as robustness or fairness) when this logic is the fruit of an opaque training process---rather than hand-coded by a human programmer. To achieve trustworthy ML, the proposed research is three-fold. First, we will shift away from the research community's focus on modifying existing ML architectures and algorithms to increase their trustworthiness. Instead, we will design them ab initio to train models that not only achieve high performance but also represent data in a different way: these representations of the data are constrained to satisfy properties (e.g., of security and privacy) that make it easier, by design, to verify that the resulting model is trustworthy. Second, our techniques will rely on these novel ML architectures to uncover the relationship between training data and predictions: this will make it easier to audit the observed behavior of a ML algorithm and obtain assurance that it is operating as expected, or when it is not, to limit the sensitivity of data collection processes to biases historically found in the data. Third, we will develop model governance mechanisms to manage ML models once they have been deployed to make predictions: this will notably inform policy discussions about intellectual property and disinformation. Our research will thus both help humans to trust ML, and ML algorithms to learn effectively and responsibly from humans. The facets of trustworthy ML that we consider are instrumental in ensuring a beneficial impact of ML, aligned with Canadian values. This will not only help with security and safety critical deployments of machine learning, such as the ones found in the cyber-physical infrastructure or the healthcare sector, it will also grow trust from the general society in applications of ML that have penetrated our daily lives: from the language models that help us type on our smartphones to the face recognition algorithms we use to board planes.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Ontario, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2020-05764
Name nom
Papernot, Nicolas
Department département
Electrical and Computer Engineering
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2021
Competition year année de concours
2020
Award amount
29000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
2803
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Knowledge representation
Research subject fr
Répresentation des connaissances
Installment
inst:2
Partie
1
Nb partie
5
Application title
Trustworthy Machine Learning by Design
Keyword
NSERC
Application summary
Machine learning (ML) brought a new computing paradigm: rather than writing programs for each problem we'd like to solve, we now write a single learning algorithm and present it with examples of inputs and outputs that are expected from the solution to the problem. This has fundamentally changed the way we build software and enabled progress in many previously challenging areas such as autonomous driving, healthcare, or computer vision. Yet, the widespread adoption of ML raises security and societal concerns that point to the lack of trustworthiness of algorithms used for training and predicting. Learning algorithms can easily be manipulated by adversaries capable of perturbing the data that ML algorithms analyze. Learning algorithms can also perpetuate and magnify biases historically found in the data they analyze. These challenges remain unaddressed despite significant interest from the research community, because it is significantly more difficult to ensure that the logic followed to solve a problem satisfies desiderata (such as robustness or fairness) when this logic is the fruit of an opaque training process---rather than hand-coded by a human programmer. To achieve trustworthy ML, the proposed research is three-fold. First, we will shift away from the research community's focus on modifying existing ML architectures and algorithms to increase their trustworthiness. Instead, we will design them ab initio to train models that not only achieve high performance but also represent data in a different way: these representations of the data are constrained to satisfy properties (e.g., of security and privacy) that make it easier, by design, to verify that the resulting model is trustworthy. Second, our techniques will rely on these novel ML architectures to uncover the relationship between training data and predictions: this will make it easier to audit the observed behavior of a ML algorithm and obtain assurance that it is operating as expected, or when it is not, to limit the sensitivity of data collection processes to biases historically found in the data. Third, we will develop model governance mechanisms to manage ML models once they have been deployed to make predictions: this will notably inform policy discussions about intellectual property and disinformation. Our research will thus both help humans to trust ML, and ML algorithms to learn effectively and responsibly from humans. The facets of trustworthy ML that we consider are instrumental in ensuring a beneficial impact of ML, aligned with Canadian values. This will not only help with security and safety critical deployments of machine learning, such as the ones found in the cyber-physical infrastructure or the healthcare sector, it will also grow trust from the general society in applications of ML that have penetrated our daily lives: from the language models that help us type on our smartphones to the face recognition algorithms we use to board planes.