Grant: Discovery Grants Program - Individual
University of Calgary
$20,000.00
Improved learning of reactive, cooperative behaviour through learning-based testing
EF 2020-21
Résumé
The long-term vision for my research is to use machine learning to improve the development of distributed systems software. My key idea is to modify and extend a concept for learning of cooperative behaviour to include the results of a learning-based testing component that finds negative consequences of the learned cooperative behaviour, including problems with intended functionality, efficiency and security.
To learn cooperative behaviour, we see the components of a distributed system as agents that can perform actions. The learning tries to create controls for these agents, so that they together achieve the functionality the whole group of agents is tasked with.
Using learning-based testing for finding negative consequences is an evolutionary learning process in which the testing component creates interaction sequences with the tested system and evaluates the sequences based on how near they come to a testing goal. The evaluation is used to learn sequencers that come nearer and nearer to creating the negative consequences the testing system is looking for.
The main research challenge is how to combine the focus of learning of cooperative behaviour on trying to deal with all possible situations the system can be in with the focus of learning-based testing on finding one particular sequence of situations revealing a particular problem of the tested system. Possible solutions of this challenge can target the way learning-based testing is performed, generalizing its results, but also modifying the general learner and the architecture with exception rules. In earlier work, we have developed the shout-ahead architecture for agents and a hybrid learning method for this architecture, combining reinforcement learning with evolutionary learning. A proof-of-concept system for developing control AIs for enemy units for the game Battle for Wesnoth showed that the best learned AIs are able to beat the rather good human-created central AI that comes with the game. We explored the usage of learning of behavior to test systems for weaknesses for several applications, including games. In all these applications, our test systems were able to find several weaknesses in the tested systems, including efficiency and security weaknesses.
Building on these works, we will explore our ideas for combining general learning of behaviour with learning-based testing in two case studies, an extension of the system for Battle for Wesnoth and an automated cattle distribution center as an example of Internet of Things applications. The anticipated result of this research is the ability to create customized open distributed systems at lower cost than current practise and with higher quality due to automization.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Alberta, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2017-04839
Name nom
Denzinger, Jörg
Department département
Computer Science
Organization id
11
Institution établissement
University of Calgary
Province en
Alberta
Province fr
Alberta
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2020
Competition year année de concours
2017
Award amount
20000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
2805
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Learning and inference theories
Research subject fr
Théories de l'apprentissage et l'inférence
Installment
inst:4-5
Partie
2
Nb partie
5
Application title
Improved learning of reactive, cooperative behaviour through learning-based testing
Keyword
NSERC
Application summary
The long-term vision for my research is to use machine learning to improve the development of distributed systems software. My key idea is to modify and extend a concept for learning of cooperative behaviour to include the results of a learning-based testing component that finds negative consequences of the learned cooperative behaviour, including problems with intended functionality, efficiency and security.
To learn cooperative behaviour, we see the components of a distributed system as agents that can perform actions. The learning tries to create controls for these agents, so that they together achieve the functionality the whole group of agents is tasked with.
Using learning-based testing for finding negative consequences is an evolutionary learning process in which the testing component creates interaction sequences with the tested system and evaluates the sequences based on how near they come to a testing goal. The evaluation is used to learn sequencers that come nearer and nearer to creating the negative consequences the testing system is looking for.
The main research challenge is how to combine the focus of learning of cooperative behaviour on trying to deal with all possible situations the system can be in with the focus of learning-based testing on finding one particular sequence of situations revealing a particular problem of the tested system. Possible solutions of this challenge can target the way learning-based testing is performed, generalizing its results, but also modifying the general learner and the architecture with exception rules. In earlier work, we have developed the shout-ahead architecture for agents and a hybrid learning method for this architecture, combining reinforcement learning with evolutionary learning. A proof-of-concept system for developing control AIs for enemy units for the game Battle for Wesnoth showed that the best learned AIs are able to beat the rather good human-created central AI that comes with the game. We explored the usage of learning of behavior to test systems for weaknesses for several applications, including games. In all these applications, our test systems were able to find several weaknesses in the tested systems, including efficiency and security weaknesses.
Building on these works, we will explore our ideas for combining general learning of behaviour with learning-based testing in two case studies, an extension of the system for Battle for Wesnoth and an automated cattle distribution center as an example of Internet of Things applications. The anticipated result of this research is the ability to create customized open distributed systems at lower cost than current practise and with higher quality due to automization.