Grant: Alliance Grants
Queen's University
$36,000.00
On Release-Readiness of ML Applications
FY 2023-24
Summary
Recent developments in software engineering for machine learning (SE4ML) have led companies around the world to invest heavily in the adoption of machine learning (ML) technology for their own products, allowing advanced classification, regression and/or clustering of users' data. However, the integration of individual ML algorithms and models into a successful software product involves substantial re-thinking of the software architectures and processes of software organizations. Today's ML world is no longer mostly about "releasing code", since development teams now need to integrate the software release pipeline with data engineering and ML model training pipelines, collaborating in cross-disciplinary fashion with ML experts, data scientists, and operators. This project aims to empirically study practices, tools, and techniques for determining release-readiness in the context of SE4ML, based on a partnership with the Artificial Intelligence (AI) Factory team of the National Bank. In close collaboration with the ML experts of our partner, we will focus on two interacting themes, i.e., (1) the automation of release-readiness decisions in an SE4ML context (including post-deployment), and (2) safe generation of anonymized regression test data based on predictions of released models (in order to better train and test future model versions). Both themes consider the impact of degrading performance of ML models (so-called "drift") on release-readiness decisions of future models. In each theme, the NSERC Alliance activities touch on more fundamental concepts and ideas, while the complementary Mitacs Accelerate internships involve a deeper validation of the activities in the context of our industry partner.
Program
Alliance Grants
Recipient
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
580832-2022
Name nom
Adams, Bram BM
Department département
Computing, School of
Organization id
29
Institution établissement
Queen's University
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2022
Award amount
36000
Program id
ALLRP
Program name en
Alliance Grants
Program name fr
Subventions Alliance
Group en
RESEARCH PARTNERSHIPS
Group fr
PARTENARIATS DE RECHERCHE
Committee code
96
Committee name en
RPP Internal Decision Cttee
Committee name fr
Cté de décision interne - PPR
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
2700
Research subject group en
Information technology
Research subject group fr
Information technology
Research subject en
Technologie de l'information
Research subject fr
Technologie de l'information
Application title
On Release-Readiness of ML Applications
Keyword
machine learning model deployment; MLOps; release engineering; release readiness
Application summary
Recent developments in software engineering for machine learning (SE4ML) have led companies around the world to invest heavily in the adoption of machine learning (ML) technology for their own products, allowing advanced classification, regression and/or clustering of users' data. However, the integration of individual ML algorithms and models into a successful software product involves substantial re-thinking of the software architectures and processes of software organizations. Today's ML world is no longer mostly about "releasing code", since development teams now need to integrate the software release pipeline with data engineering and ML model training pipelines, collaborating in cross-disciplinary fashion with ML experts, data scientists, and operators. This project aims to empirically study practices, tools, and techniques for determining release-readiness in the context of SE4ML, based on a partnership with the Artificial Intelligence (AI) Factory team of the National Bank. In close collaboration with the ML experts of our partner, we will focus on two interacting themes, i.e., (1) the automation of release-readiness decisions in an SE4ML context (including post-deployment), and (2) safe generation of anonymized regression test data based on predictions of released models (in order to better train and test future model versions). Both themes consider the impact of degrading performance of ML models (so-called "drift") on release-readiness decisions of future models. In each theme, the NSERC Alliance activities touch on more fundamental concepts and ideas, while the complementary Mitacs Accelerate internships involve a deeper validation of the activities in the context of our industry partner.