Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Discovery Grants Program - Individual

University of Calgary

$20,000.00

Testing multi-agent systems for unwanted behavior

FY 2015-16

Summary
Computer systems that adapt themselves or interact with other systems provide new challenges to software testing methods. While such systems offer the possibility for more autonomy (and less need for human administration) and the possibility for emergent behaviors that provide abilities that are not possible for a single system, these same possibilities can also result in negative and unexpected effects. A system might adapt in the wrong way, or we might have emergent misbehavior when several systems try to interact with each other. This research proposal is about new testing methods that are able to find such unwanted behavior of a system or a group of systems (a multi-agent system). This will help to ensure that we can make use of the advantages of multi-agent systems without special safety concerns. In contrast to ""conventional"" testing methods that test if a system shows the right response in given scenarios, our testing method is able to find scenarios in which a system is showing an unwanted behavior. Using our approach, we were already successful in finding unwanted behavior in the FIFA-99 soccer game (namely user actions that always resulted in scoring a goal against the computer player), in a group of rescue agents written by students in a multi-agent system class (namely creating behaviors of agents interacting with the students' agents in such a manner that the students' agents froze), and in harbour patrol and interception policies (namely behaviors for terrorist boats that resulted in them achieving a given objective despite the defenders correctly following the policies). With the proposed research program, we investigate extensions to our basic method that expand our basic process to allow for the introductions of subgoals, that allow the integration of various knowledge about testing, the application area and the tested system, and that create the possibility to have previous test runs influence the next test run. This research will enable us to develop an extended general framework for testing multi-agent systems for multi-agent specific problems together with guidelines on how to instantiate this framework for concrete multi-agent systems.
Program
Discovery Grants Program - Individual
Location
Alberta, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
238831-2010
Name nom
Denzinger, JËrg
Department département
Computer Science
Organization id
11
Institution établissement
University of Calgary
Province en
Alberta
Province fr
Alberta
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2015
Competition year année de concours
2010
Award amount
20000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
2800
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Artificial intelligence (computer vision, use2603)
Research subject fr
Intelligence artificielle (vision artificielle, utiliser 2603)
Partie
5
Nb partie
5
Application title
Testing multi-agent systems for unwanted behavior
Keyword
NSERC
Application summary
Computer systems that adapt themselves or interact with other systems provide new challenges to software testing methods. While such systems offer the possibility for more autonomy (and less need for human administration) and the possibility for emergent behaviors that provide abilities that are not possible for a single system, these same possibilities can also result in negative and unexpected effects. A system might adapt in the wrong way, or we might have emergent misbehavior when several systems try to interact with each other. This research proposal is about new testing methods that are able to find such unwanted behavior of a system or a group of systems (a multi-agent system). This will help to ensure that we can make use of the advantages of multi-agent systems without special safety concerns. In contrast to ""conventional"" testing methods that test if a system shows the right response in given scenarios, our testing method is able to find scenarios in which a system is showing an unwanted behavior. Using our approach, we were already successful in finding unwanted behavior in the FIFA-99 soccer game (namely user actions that always resulted in scoring a goal against the computer player), in a group of rescue agents written by students in a multi-agent system class (namely creating behaviors of agents interacting with the students' agents in such a manner that the students' agents froze), and in harbour patrol and interception policies (namely behaviors for terrorist boats that resulted in them achieving a given objective despite the defenders correctly following the policies). With the proposed research program, we investigate extensions to our basic method that expand our basic process to allow for the introductions of subgoals, that allow the integration of various knowledge about testing, the application area and the tested system, and that create the possibility to have previous test runs influence the next test run. This research will enable us to develop an extended general framework for testing multi-agent systems for multi-agent specific problems together with guidelines on how to instantiate this framework for concrete multi-agent systems.