Grant: Discovery Grants Program - Individual
McMaster University
$62,000.00
Information-Constrained Optimal Transport with Applications to Image Compression and Restoration
EF 2024-25
Résumé
The advent of deep learning has revolutionized numerous research areas. However, when it comes to image compression and restoration, the progress mainly comes from exploiting the data fitting capability of increasingly powerful neural networks while the fundamental principles remain essentially unchanged. Indeed, deep-learning-based image compression still largely follows the transform coding architecture with prescribed linear transforms replaced by learned nonlinear transforms; similarly, the modification of image restoration pipelines is mostly confined to substituting hand-crafted descriptors with deep feature maps. Another thing worthy of attention is that compression and restoration are often considered to be two separate problems. This separation is not compatible with many practical scenarios where corrupted images captured at one site need to be compressed and then transmitted to a different site for restoration. In such scenarios, it is clear that compression and restoration should be jointly designed to optimize the end-to-end performance. Unfortunately, the fundamental principle for this joint design is still elusive. Moreover, the employment of generative models has become increasingly popular in image compression and restoration. On the other hand, there is no systematic guidelines on how to use such models. In fact, it is a significant challenge to reconcile compression and generation in a single task. This requires a comprehensive understanding of the tension and interplay between these two seemingly conflicting goals.
The goal of this research program is to develop a new paradigm for image compression and restoration by effectively integrating information theory and transport theory. It will pave the way towards the long-term objective of developing a full-fledged information-based learning theory that allows the design and analysis of learning systems through the lens of information dynamics. The proposed research consists of three thrusts: 1) information-constrained optimal transport, 2) learned image compression, and 3) joint image compression and restoration. Specifically, the aim is to develop a theory of information-constrained optimal transport with fundamental coding theorems and efficient numerical algorithms; this theory will then be leveraged to guide the design of a refined transform coding architecture for learned image compression that can accommodate both distortion and perception constraints; it will also be used to establish a principled approach to a variety of image restoration tasks, unify the entire spectrum of learning techniques for image restoration, and direct the employment of generative models in joint image compression and restoration. The HQPs involved in this research program will receive a balanced and comprehensive training of theory building, algorithm design, and system implementation. They will be well equipped to stay at the forefront of the AI revolution.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Ontario, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2024-05695
Name nom
Chen, Jun
Department département
Electrical and Computer Engineering
Organization id
27
Institution établissement
McMaster University
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2024
Award amount
62000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1510
Committee name en
Electrical and Computer Engineering
Committee name fr
Génie électrique et informatique
Area of application code
1207
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Information, computer and communication technologies
Area of application fr
Technologie de l'information, des ordinateurs et des réseaux de communications
Research subject code
2718
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Communication and information theory
Research subject fr
Théorie des télécommunications et de l'information
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Information-Constrained Optimal Transport with Applications to Image Compression and Restoration
Keywords
deep learning; generative model; image compression; image restoration; information bottleneck; optimal transport; rate-distortion-perception theory; supervised learning; transform coding; unsupervised learning
Application summary
The advent of deep learning has revolutionized numerous research areas. However, when it comes to image compression and restoration, the progress mainly comes from exploiting the data fitting capability of increasingly powerful neural networks while the fundamental principles remain essentially unchanged. Indeed, deep-learning-based image compression still largely follows the transform coding architecture with prescribed linear transforms replaced by learned nonlinear transforms; similarly, the modification of image restoration pipelines is mostly confined to substituting hand-crafted descriptors with deep feature maps. Another thing worthy of attention is that compression and restoration are often considered to be two separate problems. This separation is not compatible with many practical scenarios where corrupted images captured at one site need to be compressed and then transmitted to a different site for restoration. In such scenarios, it is clear that compression and restoration should be jointly designed to optimize the end-to-end performance. Unfortunately, the fundamental principle for this joint design is still elusive. Moreover, the employment of generative models has become increasingly popular in image compression and restoration. On the other hand, there is no systematic guidelines on how to use such models. In fact, it is a significant challenge to reconcile compression and generation in a single task. This requires a comprehensive understanding of the tension and interplay between these two seemingly conflicting goals.
The goal of this research program is to develop a new paradigm for image compression and restoration by effectively integrating information theory and transport theory. It will pave the way towards the long-term objective of developing a full-fledged information-based learning theory that allows the design and analysis of learning systems through the lens of information dynamics. The proposed research consists of three thrusts: 1) information-constrained optimal transport, 2) learned image compression, and 3) joint image compression and restoration. Specifically, the aim is to develop a theory of information-constrained optimal transport with fundamental coding theorems and efficient numerical algorithms; this theory will then be leveraged to guide the design of a refined transform coding architecture for learned image compression that can accommodate both distortion and perception constraints; it will also be used to establish a principled approach to a variety of image restoration tasks, unify the entire spectrum of learning techniques for image restoration, and direct the employment of generative models in joint image compression and restoration. The HQPs involved in this research program will receive a balanced and comprehensive training of theory building, algorithm design, and system implementation. They will be well equipped to stay at the forefront of the AI revolution.