Canada Spends Logo

Explorez notre analyse du budget 2026

Grant: Alliance Grants

University of Calgary

$102,416.00

Managing Risks of AI-generated Code in the Software Supply Chain

EF 2024-25

Résumé
Modern software development leverages the software supply chain, by including third-party code providing external functionality into a main codebase. Traditionally, such third-party code comes in the form of functional modules (software packages), managed via a package manager, such as npm. Developers use the package manager to explicitly include modules in their codebase. AI tools for code generation - based on large language models (LLM) - constitute a novel code inclusion path within the software supply chain, one whose use is rapidly increasing. In this case, the path is implicit: LLMs are trained on large amounts of code, whose characteristics indirectly influence the generated code. LLMs hold enormous promise for simplifying code development, but they also constitute a novel threat in software supply chain security. The code in their training set is generally not vetted, containing bugs and security issues that may be picked up by the model and affect the generated code. Evidence already suggests that LLMs may learn to generate insecure or incorrect code patterns and generally perform worse - in terms of writing secure code - than humans. Further, an attacker may be able to affect model behaviours in specific situations, by poisoning the training set with crafted software samples. This state of things is concerning, as a lack of rigorous understanding of the security implications of AI-generated code is likely to prevent its use in mission-critical software infrastructure. This research program aims at ensuring model-generated code can continue to exist within the software supply chain, without generating undue risks. Our research will: (i) investigate techniques to reliably identify model-generated code, by extending code stylometry techniques and developing novel domain-specific techniques; (ii) measure the prevalence of model-generated code in the wild, further assessing its security characteristics; and (iii) develop techniques to assess the effectiveness of data poisoning attacks against LLMs, and to increase model robustness to such attacks. Overall, this research will open the way to secure code generation using LLMs.
Programme
Alliance Grants
Bénéficiaire
Emplacement
Alberta, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data

Tel que publié

Application id
580525-2022
Name nom
Tan, Peng Seng Benjamin PSB
Department département
Electrical and Software Engineering
Organization id
11
Institution établissement
University of Calgary
Province en
Alberta
Province fr
Alberta
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2022
Award amount
102416
Program id
ALLRP
Program name en
Alliance Grants
Program name fr
Subventions Alliance
Group en
RESEARCH PARTNERSHIPS
Group fr
PARTENARIATS DE RECHERCHE
Committee code
96
Committee name en
RPP Internal Decision Cttee
Committee name fr
Cté de décision interne - PPR
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
2800
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Artificial intelligence (computer vision, use2603)
Research subject fr
Intelligence artificielle (vision artificielle, utiliser 2603)
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Managing Risks of AI-generated Code in the Software Supply Chain
Keywords
adversarial ml; artificial intelligence; code classification; cybersecurity; dataset poisoning; deep neural networks; large language models; software engineering; software supply chain
Application summary
Modern software development leverages the software supply chain, by including third-party code providing external functionality into a main codebase. Traditionally, such third-party code comes in the form of functional modules (software packages), managed via a package manager, such as npm. Developers use the package manager to explicitly include modules in their codebase. AI tools for code generation - based on large language models (LLM) - constitute a novel code inclusion path within the software supply chain, one whose use is rapidly increasing. In this case, the path is implicit: LLMs are trained on large amounts of code, whose characteristics indirectly influence the generated code. LLMs hold enormous promise for simplifying code development, but they also constitute a novel threat in software supply chain security. The code in their training set is generally not vetted, containing bugs and security issues that may be picked up by the model and affect the generated code. Evidence already suggests that LLMs may learn to generate insecure or incorrect code patterns and generally perform worse - in terms of writing secure code - than humans. Further, an attacker may be able to affect model behaviours in specific situations, by poisoning the training set with crafted software samples. This state of things is concerning, as a lack of rigorous understanding of the security implications of AI-generated code is likely to prevent its use in mission-critical software infrastructure. This research program aims at ensuring model-generated code can continue to exist within the software supply chain, without generating undue risks. Our research will: (i) investigate techniques to reliably identify model-generated code, by extending code stylometry techniques and developing novel domain-specific techniques; (ii) measure the prevalence of model-generated code in the wild, further assessing its security characteristics; and (iii) develop techniques to assess the effectiveness of data poisoning attacks against LLMs, and to increase model robustness to such attacks. Overall, this research will open the way to secure code generation using LLMs.