Grant: Discovery Grants Program - Individual
Université du Québec à Montréal
$31,000.00
Towards a better use of decision trees for machine learning from small datasets
FY 2024-25
Summary
Machine learning (ML), which consists of inferring models from observations, has led to important advances in many fields. Previously constrained by limited computing power and more recently by the need to process large amounts of data, classical machine learning algorithms are designed to work quickly, thereby emphasizing computational speed over the quality and explainability of inferred models. Although potential losses in quality are generally compensated for by increasing the amount of data, there are numerous situations where it is impossible or too costly to obtain more data (e.g., in clinical studies data collection is associated with human and financial constraints that often limit the number of participants from a few dozen to a few hundred). Therefore, to meet the challenge of inferring models from small datasets, new machine learning approaches must be developed.
Long-term objective
To develop a set of new machine learning algorithms and models that are explainable, data-efficient, and provide theoretical quality guarantees.
Method
To fulfill this objective, we will design new methods based on decision trees, a well-known model that has the advantage of being explainable while providing accurate models. While in recent years much research has been conducted on optimal decision tree inference (i.e., finding a single decision tree that maximizes the probability of being the true model), in this research program we will generalize this approach by considering all possible decision trees, and averaging the classifications made by these likelihood-weighted decision trees. We propose the use of appropriate data structures, combinatorial optimization approaches and constraint solvers to address this problem efficiently.
Impact
In addition to training 2 PhD, 3 MSc and 6 BCs students in the high-demand field of ML, the proposed discovery program will advance knowledge by providing a deeper understanding of the limitations of ML in the context of small datasets. By exploring a new avenue of research to address the problem of using ML on small datasets, and by proposing practical solutions to address this challenge, this research program will lead to better use of small datasets that are currently not exploited or under-exploited by classical approaches. Therefore, all domains with small datasets, such as health domain or small organizations, which rarely have enough data to use existing ML techniques, can benefit from the knowledge that would come from carrying out the proposed research program.
Program
Discovery Grants Program - Individual
Recipient
Location
Quebec, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2023-04468
Name nom
Avellaneda, Florent
Department département
Informatique
Organization id
57
Institution établissement
Université du Québec à Montréal
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2023
Award amount
31000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
801
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Computer software
Area of application fr
Logiciel
Research subject code
2805
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Learning and inference theories
Research subject fr
Théories de l'apprentissage et l'inférence
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Towards a better use of decision trees for machine learning from small datasets
Keywords
decision tree; small data; solomonoff's induction; maxsat; sat; explainability; bayes optimal classifier
Application summary
Machine learning (ML), which consists of inferring models from observations, has led to important advances in many fields. Previously constrained by limited computing power and more recently by the need to process large amounts of data, classical machine learning algorithms are designed to work quickly, thereby emphasizing computational speed over the quality and explainability of inferred models. Although potential losses in quality are generally compensated for by increasing the amount of data, there are numerous situations where it is impossible or too costly to obtain more data (e.g., in clinical studies data collection is associated with human and financial constraints that often limit the number of participants from a few dozen to a few hundred). Therefore, to meet the challenge of inferring models from small datasets, new machine learning approaches must be developed.
Long-term objective
To develop a set of new machine learning algorithms and models that are explainable, data-efficient, and provide theoretical quality guarantees.
Method
To fulfill this objective, we will design new methods based on decision trees, a well-known model that has the advantage of being explainable while providing accurate models. While in recent years much research has been conducted on optimal decision tree inference (i.e., finding a single decision tree that maximizes the probability of being the true model), in this research program we will generalize this approach by considering all possible decision trees, and averaging the classifications made by these likelihood-weighted decision trees. We propose the use of appropriate data structures, combinatorial optimization approaches and constraint solvers to address this problem efficiently.
Impact
In addition to training 2 PhD, 3 MSc and 6 BCs students in the high-demand field of ML, the proposed discovery program will advance knowledge by providing a deeper understanding of the limitations of ML in the context of small datasets. By exploring a new avenue of research to address the problem of using ML on small datasets, and by proposing practical solutions to address this challenge, this research program will lead to better use of small datasets that are currently not exploited or under-exploited by classical approaches. Therefore, all domains with small datasets, such as health domain or small organizations, which rarely have enough data to use existing ML techniques, can benefit from the knowledge that would come from carrying out the proposed research program.