Grant: Discovery Grants Program - Individual

In this proposal, we address the general problem of extracting meaningful information from very large matrix datasets that exhibit complex correlation structures, using latent factor decomposition - a modern matrix completion techniques originally rooted in recommender systems. By "meaningful information", we refer to interpretable latent features that can be used for inference, missing data imputation or prediction purposes. Our long term goal is to propose an ensemble of statistical and computational strategies for both prediction and inference. These strategies are designed for situations where i) external information is available on the rows/columns of the data matrix, and ii) the correlation within the rows and/or columns of the data matrix exhibits some spatial, temporal, network or other graphical structure. The motivation of this research program arises from several collaborations in the fields of genetics (methylation data), medical imaging (Raman microspectroscopy data) and transportation (GPS and censor traffic data). In transportation, the (n,t)th element of the data matrix corresponds to the observed traffic flow (e.g. number of cars) passing through a sensor n located on the road network during time period t. Correlation between the rows and columns of this matrix is induced by the spatial and temporal pattern of traffic flow. In genetic methylation data, the (n,t)th element of the data matrix represents the DNA methylation level of a sample n at CpG site (DNA location) t. Such data may exhibit correlation between samples (rows of the matrix) and /or some spatial correlation between nearby CpG sites (columns of the matrix). In all these cases, existing models need to be extended to model adequately the specific characteristics of the data, the correlation structure present in the rows/columns of the data, and to integrate external covariate information when available to estimate the latent factors. Modern matrix completion techniques, originally rooted in recommender systems, have evolved to become indispensable in diverse domains, establishing themselves as a cornerstone of contemporary data science. The research program we propose addresses multiple challenges associated with the modeling of the correlation structure that underpins observed data. It provides an ideal opportunity to harness the synergy of advanced statistical modeling techniques and machine learning algorithms, thereby significantly amplifying the predictive capabilities of these models. The research conducted within this program will make valuable contributions to both statistical and machine learning literature. Moreover, each of the proposed projects is driven by practical questions stemming from ongoing collaborations in the fields of transportation and genomics. It is anticipated that the outcomes of this research will yield significant benefits for both domains.