Grant: Discovery Grants Program - Individual
York University
$19,000.00
Sparsity, thresholding and regularization in data science
FY 2022-23
Summary
Data science has brought a breakthrough in the way decisions are made in real life problems such as fraud detection, healthcare, targeted advertising, website recommendations, speech recognition, among others. Practical implementations of classic and new statistical earning techniques are the common denominator of such advances. However, most of such applications are still misunderstood by its creators, and solutions are mainly implemented out of trial and error. One of the most useful techniques in machine learning is regularization. It helps to cope with overfitting problems but also impose structures in the solution of the optimization algorithms. Thresholding estimators are a particular set of regularization estimators that impose sparse structure in the solution. Sparsity assumes that only a few covariates compose the model to explain a given response. For instance, just a few genes are relevant to explain a given disease. Moreover, sparsity can give interpretability or physical meaning to the result. The objective of this proposal is to develop theory and innovative methodologies for solving and understanding machine and statistical learning models by using sparse regularization and thresholding estimators. The proposal consists of the following three lines of research. First, high dimensional data routinely arise in econometrics, machine learning, neuroscience, and social science. I will extend my previous work in thresholding estimators to other methodologies for high dimensional data. I will be interested in applications involving categorical data for social sciences. Second, I am interested in the prediction of the risk of indirectly transmitted diseases. This can be seen as a high dimensional tomographic inverse problem. The objective is to perform an epidemiologic tomography of a region by reconstructing the areas of high and low disease risk using non-invasive measurements such as GPS animal movements, by imposing a sparse total variation spatial structure. The resulting methodology will be implemented in a full data science framework. Finally, I propose to use thresholding estimators to impose sparse structures into Deep Learning methodologies. Current deep autoencoders tend to force the architecture of a neural network. I propose to impose thresholding regularizers to jointly estimate the network architecture. I also propose a new dropout framework based on L1 regularization. Instead of randomly dropping units, I propose to perform a random selection on the regularization parameter. These methodologies involving sparsity might lead to produce better interpretation of the methods and might facilitate the derivation of mathematical properties. The success of this research program will have great contribution to the understanding of high dimensional data, machine learning, and big data, and will prompt the applications of interpretable sparse regularization in many fields in the natural sciences, social sciences, and engineering.
Program
Discovery Grants Program - Individual
Recipient
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2022-04531
Name nom
Diaz-Rodriguez, Jairo
Department département
Mathematics and Statistics - Mathematics and Statistics
Organization id
38
Institution établissement
York University
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2022
Competition year année de concours
2022
Award amount
19000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1508
Committee name en
Mathematics and Statistics
Committee name fr
Mathématiques et statistique
Area of application code
1205
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Mathematical sciences
Area of application fr
Sciences mathématiques
Research subject code
3000
Research subject group en
Statistics and probability
Research subject group fr
Statistique et probabilité
Research subject en
Statistics and probability
Research subject fr
Statistique et probabilité
Installment
inst:1
Partie
1
Nb partie
5
Application title
Sparsity, thresholding and regularization in data science
Application summary
Data science has brought a breakthrough in the way decisions are made in real life problems such as fraud detection, healthcare, targeted advertising, website recommendations, speech recognition, among others. Practical implementations of classic and new statistical earning techniques are the common denominator of such advances. However, most of such applications are still misunderstood by its creators, and solutions are mainly implemented out of trial and error. One of the most useful techniques in machine learning is regularization. It helps to cope with overfitting problems but also impose structures in the solution of the optimization algorithms. Thresholding estimators are a particular set of regularization estimators that impose sparse structure in the solution. Sparsity assumes that only a few covariates compose the model to explain a given response. For instance, just a few genes are relevant to explain a given disease. Moreover, sparsity can give interpretability or physical meaning to the result. The objective of this proposal is to develop theory and innovative methodologies for solving and understanding machine and statistical learning models by using sparse regularization and thresholding estimators. The proposal consists of the following three lines of research. First, high dimensional data routinely arise in econometrics, machine learning, neuroscience, and social science. I will extend my previous work in thresholding estimators to other methodologies for high dimensional data. I will be interested in applications involving categorical data for social sciences. Second, I am interested in the prediction of the risk of indirectly transmitted diseases. This can be seen as a high dimensional tomographic inverse problem. The objective is to perform an epidemiologic tomography of a region by reconstructing the areas of high and low disease risk using non-invasive measurements such as GPS animal movements, by imposing a sparse total variation spatial structure. The resulting methodology will be implemented in a full data science framework. Finally, I propose to use thresholding estimators to impose sparse structures into Deep Learning methodologies. Current deep autoencoders tend to force the architecture of a neural network. I propose to impose thresholding regularizers to jointly estimate the network architecture. I also propose a new dropout framework based on L1 regularization. Instead of randomly dropping units, I propose to perform a random selection on the regularization parameter. These methodologies involving sparsity might lead to produce better interpretation of the methods and might facilitate the derivation of mathematical properties. The success of this research program will have great contribution to the understanding of high dimensional data, machine learning, and big data, and will prompt the applications of interpretable sparse regularization in many fields in the natural sciences, social sciences, and engineering.