Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Discovery Grants Program - Individual

University of Waterloo

$62,000.00

Taming Deep Generative Models: Faster, Stronger and More Trustworthy

EF 2024-25

Résumé
Following the recent release of large language models (such as ChatGPT), the prominent role of generative AI has become widely recognized in academia, industry and society at large. Indeed, the ability to generate new and realistic content is often an indication that the underlying model has learned significant domain knowledge, which can be immensely useful for many downstream tasks and can address the data scarcity problem in many critical areas (such as healthcare, education, creative arts, autonomous driving). According to a survey by KPMG, 20% of Canadians are using generative AI tools on a regular basis to help them with their work or studies. Not surprisingly, an arms race on producing even bigger and better models, as well as on building new businesses to fully unlock the benefits and potentials of generative AI, is ongoing (intensely). On the flip side, modern generative AI tools remain expensive to train and deploy. Their human-level performance has also caused broad concerns, with regard to misinformation, data privacy, malicious attack, copyright protection, among others, as noted by the Canadian Centre for Cyber Security. In this proposed research program, I aim to address the above challenges facing generative AI. In particular, I will focus on the following objectives: (1) Deeper understanding between parameterization and expressiveness. I will design and compare models of generative AI in terms of their ability to take structural priori knowledge into account, and I will provide means to quantify their effectiveness in saving training data and computation. (2) Faster training and inference. I will develop novel quantization and distillation techniques to greatly reduce the training, storage and inference costs of generative AI tools, and aim to enable them on devices of varying capacity. (3) Stronger robustness against malicious adversaries. I will build analytic tools to help assess and quantify the degree of susceptibility of generative AI tools to malicious attacks, such as a small amount of poisoned data crafted and injected by a malicious adversary. I will propose new mitigation strategies to guard against the state-of-the-art attacks and apply bilevel optimization to study fundamental limitations of any attack and defense. (4) Better privacy and copyright protection. I will develop new functional differentially private training protocols to achieve an improved trade-off between generation quality, privacy and convergence speed, as well as new auditing tools to help measure and monitor privacy losses during training and deployment. I will propose new mitigation strategies to limit generative AI's direct and indirect access to copyright material. The findings from this proposed research program are expected to address important gaps in generative AI, benefit the Canadian AI ecosystem, supply and retain top AI talent for the Canadian workforce, and provide further guidance and clarity on future progress and regulation.
Programme
Discovery Grants Program - Individual
Emplacement
Ontario, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
RGPIN-2024-05121
Name nom
Yu, Yaoliang
Department département
David R. Cheriton School of Computer Science
Organization id
33
Institution établissement
University of Waterloo
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2024
Award amount
62000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
1200
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Advancement of knowledge
Area of application fr
Avancement des connaissances
Research subject code
2805
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Learning and inference theories
Research subject fr
Théories de l'apprentissage et l'inférence
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Taming Deep Generative Models: Faster, Stronger and More Trustworthy
Keywords
generative modeling; representation learning; robustness; differential privacy; model quantization; model distillation; data poisoning; fast sampling; copyright protection; optimization
Application summary
Following the recent release of large language models (such as ChatGPT), the prominent role of generative AI has become widely recognized in academia, industry and society at large. Indeed, the ability to generate new and realistic content is often an indication that the underlying model has learned significant domain knowledge, which can be immensely useful for many downstream tasks and can address the data scarcity problem in many critical areas (such as healthcare, education, creative arts, autonomous driving). According to a survey by KPMG, 20% of Canadians are using generative AI tools on a regular basis to help them with their work or studies. Not surprisingly, an arms race on producing even bigger and better models, as well as on building new businesses to fully unlock the benefits and potentials of generative AI, is ongoing (intensely). On the flip side, modern generative AI tools remain expensive to train and deploy. Their human-level performance has also caused broad concerns, with regard to misinformation, data privacy, malicious attack, copyright protection, among others, as noted by the Canadian Centre for Cyber Security. In this proposed research program, I aim to address the above challenges facing generative AI. In particular, I will focus on the following objectives: (1) Deeper understanding between parameterization and expressiveness. I will design and compare models of generative AI in terms of their ability to take structural priori knowledge into account, and I will provide means to quantify their effectiveness in saving training data and computation. (2) Faster training and inference. I will develop novel quantization and distillation techniques to greatly reduce the training, storage and inference costs of generative AI tools, and aim to enable them on devices of varying capacity. (3) Stronger robustness against malicious adversaries. I will build analytic tools to help assess and quantify the degree of susceptibility of generative AI tools to malicious attacks, such as a small amount of poisoned data crafted and injected by a malicious adversary. I will propose new mitigation strategies to guard against the state-of-the-art attacks and apply bilevel optimization to study fundamental limitations of any attack and defense. (4) Better privacy and copyright protection. I will develop new functional differentially private training protocols to achieve an improved trade-off between generation quality, privacy and convergence speed, as well as new auditing tools to help measure and monitor privacy losses during training and deployment. I will propose new mitigation strategies to limit generative AI's direct and indirect access to copyright material. The findings from this proposed research program are expected to address important gaps in generative AI, benefit the Canadian AI ecosystem, supply and retain top AI talent for the Canadian workforce, and provide further guidance and clarity on future progress and regulation.