Grant: Discovery Grants Program - Individual
University of Toronto
$92,000.00
Encoded computing for efficient and robust large-scale distributed optimization
EF 2023-24
Résumé
The availability of big data and the use of these data in training artificial intelligence (AI) systems is changing the way companies do business in a wide swath of industries, from finance to entertainment to drug discovery. These advances rely on a computing, networking, and data storage infrastructure, the growing capabilities of which are key to realizing the promise of AI. However, as data sets and processing requirements scale up massively, classic single-processor computing paradigms cannot keep up. There has therefore been a renaissance in understanding how to bring large-scale parallelization to bear on algorithmic design for learning-specific workloads. As we outline in this proposal, we can draw on techniques, perspectives, and solutions from digital communications and error-correction coding to develop novel robust and resource-efficient approaches to parallelized computing. Our long-term vision is that by drawing on the theory and practice of digital communications we can deliver novel and unexpected impact on the theory and practice of computation. In the shorter term, we will build towards our vision by following three coupled research themes. These themes draw on and contribute to the theory and practice of error-correction coding and distributed optimization to enable large-scale, robust, and efficient computing systems. In the first theme we design large-scale distributed optimization techniques that deliver idealized performance while coping with the non-idealities of real-world cloud computing systems including the effects of "straggler" nodes and network delays. In the second, we draw on perspectives from information theory and error correction to develop novel "coding-for-computing" paradigms that deliver robust and efficient computation. In the final theme we work closely with systems groups to help push our results into application and to improve our understanding of the myriad real-world issues facing computing systems, motivating further research and deepening collaborations.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Ontario, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2019-05828
Name nom
Draper, Stark
Department département
Electrical and Computer Engineering
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2019
Award amount
92000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1510
Committee name en
Electrical and Computer Engineering
Committee name fr
Génie électrique et informatique
Area of application code
1207
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Information, computer and communication technologies
Area of application fr
Technologie de l'information, des ordinateurs et des réseaux de communications
Research subject code
2715
Research subject group en
Information technology
Research subject group fr
Optimization
Research subject en
Technologie de l'information
Research subject fr
Optimisation
Application title
Encoded computing for efficient and robust large-scale distributed optimization
Keyword
Artificial intelligence; Distributed computing; Error correction coding; Joint source channel coding; Optimization; Systems and compute architecture
Application summary
The availability of big data and the use of these data in training artificial intelligence (AI) systems is changing the way companies do business in a wide swath of industries, from finance to entertainment to drug discovery. These advances rely on a computing, networking, and data storage infrastructure, the growing capabilities of which are key to realizing the promise of AI. However, as data sets and processing requirements scale up massively, classic single-processor computing paradigms cannot keep up. There has therefore been a renaissance in understanding how to bring large-scale parallelization to bear on algorithmic design for learning-specific workloads. As we outline in this proposal, we can draw on techniques, perspectives, and solutions from digital communications and error-correction coding to develop novel robust and resource-efficient approaches to parallelized computing. Our long-term vision is that by drawing on the theory and practice of digital communications we can deliver novel and unexpected impact on the theory and practice of computation. In the shorter term, we will build towards our vision by following three coupled research themes. These themes draw on and contribute to the theory and practice of error-correction coding and distributed optimization to enable large-scale, robust, and efficient computing systems. In the first theme we design large-scale distributed optimization techniques that deliver idealized performance while coping with the non-idealities of real-world cloud computing systems including the effects of "straggler" nodes and network delays. In the second, we draw on perspectives from information theory and error correction to develop novel "coding-for-computing" paradigms that deliver robust and efficient computation. In the final theme we work closely with systems groups to help push our results into application and to improve our understanding of the myriad real-world issues facing computing systems, motivating further research and deepening collaborations.