Grant: Discovery Grants Program - Individual
University of Ottawa
$6,694.00
Dealing with Extreme Class Imbalance Learning in Defense and Security Applications
FY 2016-17
Summary
Defense and Security applications, such as threat monitoring, e.g., the detection of hazardous atmospheric emissions, underwater mines, or computer network attacks, must all deal with the same underpinning problem: the extreme scarcity and disparity of data describing an event in need of detection. This creates a significant challenge for machine learning applications. Sampling methods have aimed to increase the amount of event data, but in extreme cases in which only a handful of event data are available, current methods of this kind have not yet been successful. The defense and security community's approach has typically been to generate simulated data, based on domain knowledge, to 'complement' the missing event data. However, given the usual simplicity of the simulation models, this type of response is generally unsatisfactory. Another machine learning response has been to use one-class learning (outlier detection) approaches to model background data, which is typically plentiful, and to send a signal when a 'suspected' outlier is encountered. However, current approaches of this kind are typically much less powerful than their binary-class counterparts.
The research program that I propose in this Discovery Grant application will effectively address the extreme class imbalance problem. In particular, I propose a new approach called Negative Learning. Negative Learning consists of recognizing that although we do not have enough instances of abnormal/threat data, we have many instances of normal/background ones. Based on this fact, we consider the abnormal class to be any and all instances missing from the normal class, and we propose to appropriately sample from this ""negative of the normal class"". The technique will be based on the following generative approach composed of four steps.
In the first step, a probability density function will be derived from the background data. In the second step, new event data will be artificially generated in the low probability regions of that model, because data in those regions can be thought of as borderline instances of the abnormal class. This means that unlike in simple methods like the Synthetic Minority Over-sampling Technique (SMOTE) approach, which only generates new samples within the convex hull described by the available data, I propose to generate data far beyond the convex hull of the available ones. The third step will consist of refining the artificial data set produced by the first and second step using domain knowledge and user guidance. In particular, I will use active learning and its derivatives as well as domain knowledge integration to augment the abnormal class with data not generated by the first two steps, and to trim it by eliminating redundant or implausible data. The last step of the process will be the application of binary classifiers to the newly generated data set and the evaluation of the overall system as compared to other techniques currently available (binary- and one-class based).
This research should have a significant impact on Defense and Security because it will make the application of Machine Learning techniques to the problems encountered in the field much more realistic. The techniques developed for this purpose will also find applications in other fields such as in the medical domain and in text mining.
This research will allow two Ph.D. and three Master's student to study under my supervision and carry out their studies from beginning to end.
Program
Discovery Grants Program - Individual
Recipient
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2014-04889
Name nom
Japkowicz, Nathalie
Department département
Electrical Engineering and Computer Science , School of
Organization id
28
Institution établissement
University of Ottawa
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2016
Competition year année de concours
2014
Award amount
6694
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2805
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Learning and inference theories
Research subject fr
Théories de l'apprentissage et l'inférence
Partie
3
Nb partie
5
Application title
Dealing with Extreme Class Imbalance Learning in Defense and Security Applications
Keyword
NSERC
Application summary
Defense and Security applications, such as threat monitoring, e.g., the detection of hazardous atmospheric emissions, underwater mines, or computer network attacks, must all deal with the same underpinning problem: the extreme scarcity and disparity of data describing an event in need of detection. This creates a significant challenge for machine learning applications. Sampling methods have aimed to increase the amount of event data, but in extreme cases in which only a handful of event data are available, current methods of this kind have not yet been successful. The defense and security community's approach has typically been to generate simulated data, based on domain knowledge, to 'complement' the missing event data. However, given the usual simplicity of the simulation models, this type of response is generally unsatisfactory. Another machine learning response has been to use one-class learning (outlier detection) approaches to model background data, which is typically plentiful, and to send a signal when a 'suspected' outlier is encountered. However, current approaches of this kind are typically much less powerful than their binary-class counterparts.
The research program that I propose in this Discovery Grant application will effectively address the extreme class imbalance problem. In particular, I propose a new approach called Negative Learning. Negative Learning consists of recognizing that although we do not have enough instances of abnormal/threat data, we have many instances of normal/background ones. Based on this fact, we consider the abnormal class to be any and all instances missing from the normal class, and we propose to appropriately sample from this ""negative of the normal class"". The technique will be based on the following generative approach composed of four steps.
In the first step, a probability density function will be derived from the background data. In the second step, new event data will be artificially generated in the low probability regions of that model, because data in those regions can be thought of as borderline instances of the abnormal class. This means that unlike in simple methods like the Synthetic Minority Over-sampling Technique (SMOTE) approach, which only generates new samples within the convex hull described by the available data, I propose to generate data far beyond the convex hull of the available ones. The third step will consist of refining the artificial data set produced by the first and second step using domain knowledge and user guidance. In particular, I will use active learning and its derivatives as well as domain knowledge integration to augment the abnormal class with data not generated by the first two steps, and to trim it by eliminating redundant or implausible data. The last step of the process will be the application of binary classifiers to the newly generated data set and the evaluation of the overall system as compared to other techniques currently available (binary- and one-class based).
This research should have a significant impact on Defense and Security because it will make the application of Machine Learning techniques to the problems encountered in the field much more realistic. The techniques developed for this purpose will also find applications in other fields such as in the medical domain and in text mining.
This research will allow two Ph.D. and three Master's student to study under my supervision and carry out their studies from beginning to end.