Grant: Alliance Grants
University of Calgary
$50,000.00
Enhancing cybersecurity through advanced reasoning in large language models
EF 2024-25
Résumé
Our project addresses the growing need for enhanced cybersecurity in industrial applications by improving the reasoning capabilities of Large Language Models (LLMs). This research is vital for Alberta's key industries-energy, healthcare, financial services, and agriculture-that increasingly rely on AI systems. Enhancing LLM security protects these sectors from cyber threats, safeguarding critical infrastructure and sensitive data. As LLMs are increasingly used across sectors, their susceptibility to cyberattacks-such as prompt injection, data poisoning, model extraction, and adversarial examples poses serious security challenges. By advancing reasoning abilities and incorporating formal methods, our goal is to develop LLMs that are more interpretable, capable of explaining their outputs, and resilient against cyber threats. Key activities involve creating synthetic datasets for reasoning tasks, employing efficient machine learning techniques like fine-tuning on formal reasoning and natural language, and applying reinforcement learning with proof assistants to refine LLM reasoning. All models, data, and methods will be open-source, fostering collaboration and accelerating progress.
Our interdisciplinary team, comprising experts from the University of Alberta (UAlberta) and the University of Calgary (UCalgary), specializes in formalized mathematics, machine learning, and cybersecurity. The project aims to develop LLMs with superior reasoning and explainability, establish benchmarks for reasoning-enhanced LLMs in cybersecurity, and deliver robust models resistant to attacks.
Anticipated outcomes include significant advancements in LLM reasoning, enhanced cybersecurity for industrial applications, and improved interpretability of LLM decision-making. We aim to position Alberta as a leader in AI-enhanced, cybersecurity-aware solutions, and to train highly qualified personnel in AI, reasoning, and cybersecurity, thereby enriching the region's talent pool in emerging technologies. Our project will utilize compute resources from UAlberta, UCalgary, and the Digital Research Alliance of Canada.
Programme
Alliance Grants
Bénéficiaire
Emplacement
Alberta, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
600038-2024
Name nom
Bitoun, Thomas Charles Maurice TCM
Department département
Mathematics and Statistics
Organization id
11
Institution établissement
University of Calgary
Province en
Alberta
Province fr
Alberta
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2024
Award amount
50000
Program id
ALLRP
Program name en
Alliance Grants
Program name fr
Subventions Alliance
Group en
RESEARCH PARTNERSHIPS
Group fr
PARTENARIATS DE RECHERCHE
Committee code
96
Committee name en
RPP Internal Decision Cttee
Committee name fr
Cté de décision interne - PPR
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
2800
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Artificial intelligence (computer vision, use2603)
Research subject fr
Intelligence artificielle (vision artificielle, utiliser 2603)
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Enhancing cybersecurity through advanced reasoning in large language models
Keywords
cybersecurity; reasoning
Application summary
Our project addresses the growing need for enhanced cybersecurity in industrial applications by improving the reasoning capabilities of Large Language Models (LLMs). This research is vital for Alberta's key industries-energy, healthcare, financial services, and agriculture-that increasingly rely on AI systems. Enhancing LLM security protects these sectors from cyber threats, safeguarding critical infrastructure and sensitive data. As LLMs are increasingly used across sectors, their susceptibility to cyberattacks-such as prompt injection, data poisoning, model extraction, and adversarial examples poses serious security challenges. By advancing reasoning abilities and incorporating formal methods, our goal is to develop LLMs that are more interpretable, capable of explaining their outputs, and resilient against cyber threats. Key activities involve creating synthetic datasets for reasoning tasks, employing efficient machine learning techniques like fine-tuning on formal reasoning and natural language, and applying reinforcement learning with proof assistants to refine LLM reasoning. All models, data, and methods will be open-source, fostering collaboration and accelerating progress.
Our interdisciplinary team, comprising experts from the University of Alberta (UAlberta) and the University of Calgary (UCalgary), specializes in formalized mathematics, machine learning, and cybersecurity. The project aims to develop LLMs with superior reasoning and explainability, establish benchmarks for reasoning-enhanced LLMs in cybersecurity, and deliver robust models resistant to attacks.
Anticipated outcomes include significant advancements in LLM reasoning, enhanced cybersecurity for industrial applications, and improved interpretability of LLM decision-making. We aim to position Alberta as a leader in AI-enhanced, cybersecurity-aware solutions, and to train highly qualified personnel in AI, reasoning, and cybersecurity, thereby enriching the region's talent pool in emerging technologies. Our project will utilize compute resources from UAlberta, UCalgary, and the Digital Research Alliance of Canada.