Grant: Discovery Grants Program - Individual
McMaster University
$25,000.00
Algorithmic Watchdog for Differential Privacy: From Theory to Practice
FY 2022-23
Summary
Armed with powerful advances in machine learning (ML), the ability of an adversary to gather personal information from an individual's expanding digital footprint is outstripping anyone's capability to keep their information private. While the data collected can have tremendous benefit for consumers via technologies built on ML, this benefit must be tempered with meaningful assurances of privacy. The de-facto standard for reasoning about such assurances in ML is differential privacy (DP). However, despite its widespread adoption in governments and corporations, there is no standardized approach for evaluating and monitoring DP technologies. Incorrectly specified DP parameters may incur a prohibitively high utility loss and-at worse-not provide privacy against an adversary. Today, these potential harms are silent: there are no automated methods for tuning and monitoring algorithms for DP misuse. This research fills this gap by creating mathematical methods that precisely characterize the risks of private information leakage and reduced utility in existing DP algorithms. These methods will constitute a rigorous blueprint for scalable "algorithmic watchdogs" that monitor DP technologies for misuse and unintended harm. Algorithmic watchdogs will reduce the potential harm of deploying DP in applications that use individual-level sensitive data such as Canada's census records. Moreover, they will fundamentally impact how DP is implemented by governments (e.g., Statistics Canada) and companies, and help developers optimally tune the parameters of private learning algorithms, monitor DP technologies for performance loss, and alert for potential misuse. The proposed work has two key novelties. First, we propose to jointly characterize the "operational privacy" and utility guarantees of existing DP algorithms by applying powerful tools from information theory. These two aspects must be simultaneously tracked to avoid misuse in DP deployments. The key advantage of the information-theoretic approach is that it will mathematically delineate the fundamental limits of private learning in an algorithmic-independent manner. These limits, in turn, serve as a rigorous blueprint for designing and benchmarking practical algorithms. Second, the research cuts across the DP development stack: it prevents misuse starting from the (many) mathematical definitions of DP to the actual deployment of the technology using open-source DP software. The overall long-term goal of this initiative is to provide data scientists in governments and corporations with a set of theoretically-grounded and algorithmic tools to guarantee meaningful and operational privacy with provably minimal unintended harms. This research will help government's data scientists understand how operational privacy can impact the accuracy of ML tasks, as well as how to optimally select parameters of learning algorithms in two popular DP applications: queries to statistical databases and training ML models.
Program
Discovery Grants Program - Individual
Recipient
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2022-05283
Name nom
Asoodeh, Shahab
Department département
Computing and Software - Computing and Software
Organization id
27
Institution établissement
McMaster University
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2022
Competition year année de concours
2022
Award amount
25000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
805
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Communications technologies (satellites, radar, etc.)
Area of application fr
Technologie des télécommunications (satellites, radar,etc.)
Research subject code
2800
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Artificial intelligence (computer vision, use2603)
Research subject fr
Intelligence artificielle (vision artificielle, utiliser 2603)
Installment
inst:1
Partie
1
Nb partie
5
Application title
Algorithmic Watchdog for Differential Privacy: From Theory to Practice
Application summary
Armed with powerful advances in machine learning (ML), the ability of an adversary to gather personal information from an individual's expanding digital footprint is outstripping anyone's capability to keep their information private. While the data collected can have tremendous benefit for consumers via technologies built on ML, this benefit must be tempered with meaningful assurances of privacy. The de-facto standard for reasoning about such assurances in ML is differential privacy (DP). However, despite its widespread adoption in governments and corporations, there is no standardized approach for evaluating and monitoring DP technologies. Incorrectly specified DP parameters may incur a prohibitively high utility loss and-at worse-not provide privacy against an adversary. Today, these potential harms are silent: there are no automated methods for tuning and monitoring algorithms for DP misuse. This research fills this gap by creating mathematical methods that precisely characterize the risks of private information leakage and reduced utility in existing DP algorithms. These methods will constitute a rigorous blueprint for scalable "algorithmic watchdogs" that monitor DP technologies for misuse and unintended harm. Algorithmic watchdogs will reduce the potential harm of deploying DP in applications that use individual-level sensitive data such as Canada's census records. Moreover, they will fundamentally impact how DP is implemented by governments (e.g., Statistics Canada) and companies, and help developers optimally tune the parameters of private learning algorithms, monitor DP technologies for performance loss, and alert for potential misuse. The proposed work has two key novelties. First, we propose to jointly characterize the "operational privacy" and utility guarantees of existing DP algorithms by applying powerful tools from information theory. These two aspects must be simultaneously tracked to avoid misuse in DP deployments. The key advantage of the information-theoretic approach is that it will mathematically delineate the fundamental limits of private learning in an algorithmic-independent manner. These limits, in turn, serve as a rigorous blueprint for designing and benchmarking practical algorithms. Second, the research cuts across the DP development stack: it prevents misuse starting from the (many) mathematical definitions of DP to the actual deployment of the technology using open-source DP software. The overall long-term goal of this initiative is to provide data scientists in governments and corporations with a set of theoretically-grounded and algorithmic tools to guarantee meaningful and operational privacy with provably minimal unintended harms. This research will help government's data scientists understand how operational privacy can impact the accuracy of ML tasks, as well as how to optimally select parameters of learning algorithms in two popular DP applications: queries to statistical databases and training ML models.