Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Discovery Grants Program - Individual

Université de Montréal

$46,000.00

Machine Learning with Tensor Networks

FY 2024-25

Summary
Machine learning (ML) is a rapidly growing field getting adopted in a wide range of technological and scientific areas. Recently, the ML community has shown an increasing interest towards tensors. The most obvious connection between tensors and ML appears when the data has a natural tensor structure (e.g. videos), but tensors and multilinear algebra can also arise as powerful tools to design efficient and scalable learning algorithms able to capture high-order dependencies in data. While linear algebra is at the backbone of many ML algorithms, recent progresses in multilinear algebra and tensor decomposition techniques open the door to developing a new generation of learning tools able to leverage multi-modal structure and scale up existing algorithms to high-dimensional data. One of my long term research objectives is to bring multilinear algebra and tensor decomposition techniques to the status of linear algebra as one of the backbones of ML theory and algorithms. Similarly to matrix factorization techniques, multilinear methods rely on decomposing high-order tensors into small pieces. However, whereas there is a unique way to define the rank of a matrix, several definitions of tensor rank are commonly used. Similarly to the matrix case, where the rank of a matrix is intrinsically tied with how it can be factorized, each notion of tensor rank arises from a particular way to decompose a tensor into smaller blocks. Tensor networks offer an intuitive way to visually depict such decompositions of tensors. In this formalism, each tensor decomposition model corresponds to a particular graph topology and each graph topology gives rise to a particular tensor decomposition model and to an associated notion of rank. The objective of this research program is the elaboration of a unifying framework relying on the tensor network formalism to study the connections between the different tensor decomposition formats and to design learning algorithms able to combine the power of different decomposition models. This will be achieved through the following four research axis: 1) Provide a theoretical analysis of how different tensor decomposition models induce distinct forms of regularization and compression in learning algorithms. 2) Leverage the tensor network formalism to go beyond classical decomposition models and develop a unified framework to reason about tensor factorization in ML. 3) Develop adaptive algorithms able to find the tensor network structure best suited to a particular task. 4) Demonstrate the effectiveness and applicability of the aforementioned theoretical framework and algorithms in application domains. This research program will help spread the adoption of tensor methods in scientific and technology areas where high-dimensional multi-modal data is ubiquitous, such as neuro-science. It will also have a significant impact on the ML community by providing a unified view of multilinear regularization and tensor decomposition techniques.
Program
Discovery Grants Program - Individual
Location
Quebec, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
RGPIN-2019-05949
Name nom
RABUSSEAU, Guillaume
Department département
DIRO
Organization id
63
Institution établissement
Université de Montréal
Province en
Québec
Province fr
Québec
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2019
Award amount
46000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
1205
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Mathematical sciences
Area of application fr
Sciences mathématiques
Research subject code
2805
Research subject group en
Artificial intelligence
Research subject group fr
Intelligence artificielle
Research subject en
Learning and inference theories
Research subject fr
Théories de l'apprentissage et l'inférence
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Machine Learning with Tensor Networks
Keywords
machine learning; multilinear algebra; tensors; tensor decomposition techniques; tensor networks; large scale algorithms
Application summary
Machine learning (ML) is a rapidly growing field getting adopted in a wide range of technological and scientific areas. Recently, the ML community has shown an increasing interest towards tensors. The most obvious connection between tensors and ML appears when the data has a natural tensor structure (e.g. videos), but tensors and multilinear algebra can also arise as powerful tools to design efficient and scalable learning algorithms able to capture high-order dependencies in data. While linear algebra is at the backbone of many ML algorithms, recent progresses in multilinear algebra and tensor decomposition techniques open the door to developing a new generation of learning tools able to leverage multi-modal structure and scale up existing algorithms to high-dimensional data. One of my long term research objectives is to bring multilinear algebra and tensor decomposition techniques to the status of linear algebra as one of the backbones of ML theory and algorithms. Similarly to matrix factorization techniques, multilinear methods rely on decomposing high-order tensors into small pieces. However, whereas there is a unique way to define the rank of a matrix, several definitions of tensor rank are commonly used. Similarly to the matrix case, where the rank of a matrix is intrinsically tied with how it can be factorized, each notion of tensor rank arises from a particular way to decompose a tensor into smaller blocks. Tensor networks offer an intuitive way to visually depict such decompositions of tensors. In this formalism, each tensor decomposition model corresponds to a particular graph topology and each graph topology gives rise to a particular tensor decomposition model and to an associated notion of rank. The objective of this research program is the elaboration of a unifying framework relying on the tensor network formalism to study the connections between the different tensor decomposition formats and to design learning algorithms able to combine the power of different decomposition models. This will be achieved through the following four research axis: 1) Provide a theoretical analysis of how different tensor decomposition models induce distinct forms of regularization and compression in learning algorithms. 2) Leverage the tensor network formalism to go beyond classical decomposition models and develop a unified framework to reason about tensor factorization in ML. 3) Develop adaptive algorithms able to find the tensor network structure best suited to a particular task. 4) Demonstrate the effectiveness and applicability of the aforementioned theoretical framework and algorithms in application domains. This research program will help spread the adoption of tensor methods in scientific and technology areas where high-dimensional multi-modal data is ubiquitous, such as neuro-science. It will also have a significant impact on the ML community by providing a unified view of multilinear regularization and tensor decomposition techniques.