Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Alliance Grants

University of Toronto

$45,238.00

Hierarchical Distributed Machine Learning

FY 2023-24

Summary
Distributed Machine Learning is an emerging research area where user data cannot be centrally located at a single site but must be distributed across multiple devices that must cooperatively participate in the model training and inference tasks. In recent years there has been a significant interest from both the industry and academia to address various challenges in this area and a number of open source software libraries that provide support for distributed training and prediction have been developed. Nevertheless, a number of important challenges still remain to be addressed. In current systems the user nodes transmit raw gradient vectors computed on the locally available training data, which can leak sensitive information. Current solutions that encrypt the gradient vectors are based on secure multi-party computation techniques that have high communication complexity and do not scale to a large number of participating users. Secondly the models that should be trained may consist of millions of parameters and the transmission of raw gradient vectors would be bandwidth intensive. The proposed research will address these challenges by following two key approaches: (1) apply principles from cryptography to develop privacy preserving methods and (2) develop hierarchical clustering among users to improve efficiency. Our research methodology will also incorporate fairness criteria i.e., by design, variates such as sex, demographics, gender, race, ethnicity and other factors will not influence the decision making of the proposed algorithms. In collaboration with our industry partners --- Hitachi Solutions and Filament AI --- the proposed research will lead to real-world impact on a number of sectors including industrial automation, healthcare, self-driving cars and smart buildings. The HQP involved in the research project will be a diverse team that will be actively involved in a number of outreach activities promoting EDI initiatives, be trained in cutting edge research in machine learning and communication systems and will closely interact with both the industry partners.
Program
Alliance Grants
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
580546-2022
Name nom
Khisti, Ashish AJ
Department département
Electrical & Computer Engineering, Edward S. Rogers Sr. Department of
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2022
Award amount
45238
Program id
ALLRP
Program name en
Alliance Grants
Program name fr
Subventions Alliance
Group en
RESEARCH PARTNERSHIPS
Group fr
PARTENARIATS DE RECHERCHE
Committee code
96
Committee name en
RPP Internal Decision Cttee
Committee name fr
Cté de décision interne - PPR
Area of application code
1207
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Information, computer and communication technologies
Area of application fr
Technologie de l'information, des ordinateurs et des réseaux de communications
Research subject code
9999
Research subject group en
Not available
Research subject group fr
Not available
Research subject en
Non disponible
Research subject fr
Non disponible
Application title
Hierarchical Distributed Machine Learning
Keyword
Data Compression; Fairness; Federated Learning; Homomorphic Encryption; Privacy Preserving Machine Learning; Secure aggregation; Secure Multiparty Computation
Application summary
Distributed Machine Learning is an emerging research area where user data cannot be centrally located at a single site but must be distributed across multiple devices that must cooperatively participate in the model training and inference tasks. In recent years there has been a significant interest from both the industry and academia to address various challenges in this area and a number of open source software libraries that provide support for distributed training and prediction have been developed. Nevertheless, a number of important challenges still remain to be addressed. In current systems the user nodes transmit raw gradient vectors computed on the locally available training data, which can leak sensitive information. Current solutions that encrypt the gradient vectors are based on secure multi-party computation techniques that have high communication complexity and do not scale to a large number of participating users. Secondly the models that should be trained may consist of millions of parameters and the transmission of raw gradient vectors would be bandwidth intensive. The proposed research will address these challenges by following two key approaches: (1) apply principles from cryptography to develop privacy preserving methods and (2) develop hierarchical clustering among users to improve efficiency. Our research methodology will also incorporate fairness criteria i.e., by design, variates such as sex, demographics, gender, race, ethnicity and other factors will not influence the decision making of the proposed algorithms. In collaboration with our industry partners --- Hitachi Solutions and Filament AI --- the proposed research will lead to real-world impact on a number of sectors including industrial automation, healthcare, self-driving cars and smart buildings. The HQP involved in the research project will be a diverse team that will be actively involved in a number of outreach activities promoting EDI initiatives, be trained in cutting edge research in machine learning and communication systems and will closely interact with both the industry partners.