Grant: Discovery Grants Program - Individual
The University of British Columbia
$23,000.00
Statistical machine learning for dependent data: symmetry and novel dependence structures
FY 2024-25
Summary
Long-term objective, methods and HQP
My long-term objective is to develop statistical and machine learning (ML) methods that maximize information extracted from data by striking a balance between model and computational complexities, and to develop theory that characterizes the balance. Symmetry plays a key role, acting to separate relevant and irrelevant statistical information. Methodological and theoretical aspects are motivated by applications in network analysis, evolutionary processes such as cancer phylogenetics, probabilistic programming systems, and by problems arising in deep learning and ML. Training HQP, particularly graduate students, is a top priority; using equitable recruiting and mentoring practices, I intend to train a diverse group of at least 14 HQP over the funding period.
Review and recent progress
Many problems in modern statistics and ML require finding the appropriate balance between a high-fidelity model of data and the complexity of computation required to perform inference and make predictions. Statistical independence is a hallmark of efficient computational techniques like parallelization and stochastic optimization, and too much dependence between model components will lead to intractable inference methods. Conversely, models that do not allow for enough dependence discard crucial statistical information. At present, our most widely used methods rely on a small set of simple dependence structures, and models with more complex forms of dependence face difficult or impossible inference objectives. My recent work has addressed this tradeoff in network data analysis and in deep learning, using symmetry and so-called neutral-to-the-left (NTL) stochastic processes.
Short-term objectives and impact
Building on my recent progress, I have three short-term and impactful objectives.
1) Develop probabilistic models, inference, and applications based on novel dependence structures. We will emphasize NTL processes, which are ideal Bayesian priors for data with unobserved evolution. This objective will generate theory, methods, and open-source software to make these processes useful for practitioners in fields ranging broadly from cancer phylogenetics to record linkage.
2) Create theory and methods for efficient computational techniques based on non-traditional symmetries. A small set of symmetries, such as permutation-invariance, are relied on for core computational techniques in ML. This objective will extend existing methods and generate new ones, based on symmetries arising in my recent work and that of others.
3) Understand the relationship between conditional independence, symmetry, and computation. The statistical utility of conditional independence is well-known. Similarly, many efficient computational methods rely on conditional independence. Symmetry is a natural bridge between the two. This objective will extend our understanding of the fundamental relationships between conditioning, symmetry, and computation.
Program
Discovery Grants Program - Individual
Recipient
Location
British Columbia, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
RGPIN-2020-04995
Name nom
Bloem-Reddy, Benjamin
Department département
Statistics
Organization id
2
Institution établissement
The University of British Columbia
Province en
British Columbia
Province fr
Colombie-Britannique
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2020
Award amount
23000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1508
Committee name en
Mathematics and Statistics
Committee name fr
Mathématiques et statistique
Area of application code
1207
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Information, computer and communication technologies
Area of application fr
Technologie de l'information, des ordinateurs et des réseaux de communications
Research subject code
3000
Research subject group en
Statistics and probability
Research subject group fr
Statistique et probabilité
Research subject en
Statistics and probability
Research subject fr
Statistique et probabilité
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Statistical machine learning for dependent data: symmetry and novel dependence structures
Keywords
probabilistic machine learning; symmetry; bayesian nonparametrics; statistical learning theory; computational statistics; network analysis; deep learning; stochastic processes; bayesian inference; probabilistic programming
Application summary
Long-term objective, methods and HQP
My long-term objective is to develop statistical and machine learning (ML) methods that maximize information extracted from data by striking a balance between model and computational complexities, and to develop theory that characterizes the balance. Symmetry plays a key role, acting to separate relevant and irrelevant statistical information. Methodological and theoretical aspects are motivated by applications in network analysis, evolutionary processes such as cancer phylogenetics, probabilistic programming systems, and by problems arising in deep learning and ML. Training HQP, particularly graduate students, is a top priority; using equitable recruiting and mentoring practices, I intend to train a diverse group of at least 14 HQP over the funding period.
Review and recent progress
Many problems in modern statistics and ML require finding the appropriate balance between a high-fidelity model of data and the complexity of computation required to perform inference and make predictions. Statistical independence is a hallmark of efficient computational techniques like parallelization and stochastic optimization, and too much dependence between model components will lead to intractable inference methods. Conversely, models that do not allow for enough dependence discard crucial statistical information. At present, our most widely used methods rely on a small set of simple dependence structures, and models with more complex forms of dependence face difficult or impossible inference objectives. My recent work has addressed this tradeoff in network data analysis and in deep learning, using symmetry and so-called neutral-to-the-left (NTL) stochastic processes.
Short-term objectives and impact
Building on my recent progress, I have three short-term and impactful objectives.
1) Develop probabilistic models, inference, and applications based on novel dependence structures. We will emphasize NTL processes, which are ideal Bayesian priors for data with unobserved evolution. This objective will generate theory, methods, and open-source software to make these processes useful for practitioners in fields ranging broadly from cancer phylogenetics to record linkage.
2) Create theory and methods for efficient computational techniques based on non-traditional symmetries. A small set of symmetries, such as permutation-invariance, are relied on for core computational techniques in ML. This objective will extend existing methods and generate new ones, based on symmetries arising in my recent work and that of others.
3) Understand the relationship between conditional independence, symmetry, and computation. The statistical utility of conditional independence is well-known. Similarly, many efficient computational methods rely on conditional independence. Symmetry is a natural bridge between the two. This objective will extend our understanding of the fundamental relationships between conditioning, symmetry, and computation.