Grant: Discovery Grants Program - Individual

Video compression (VC) is one of the key technologies that enables streaming, downloading, and broadcasting of video content over wireless networks. The important purpose of VC is to minimize the amount of data required to capture the content, without compromising the visual quality. Given the sheer amounts of data associated with most video content, it is practically impossible to send high quality live video over even the fastest wireless link, without first applying very efficient compression to reduce the required network bandwidth. While the capacity of wireless networks will continue to grow, so will the raw bitrates of emerging applications such as 8K HD video, high dynamic range (HDR) and wide color gamut (WCG) video etc. Cisco Systems Inc. estimates that, by 2021, over 80% of the Internet traffic would be video. Many in the industry are of the opinion that substantial improvements in video compression efficiency can no longer be achieved without fundamental changes to the conventional motion compensated transform coding (MCTC) framework on which the current VC technology is built. This research program will investigate fundamentally new disruptive VC technologies which can help go beyond the performance limits of the MCTC. The proposed research departs from the established practice in two major ways with the potential to achieve order-of-magnitude improvements in video compression efficiency. First,  it aims to forgo the conventional approach of putting together a series of processing operations in a heuristic manner to implement complex functions such as motion compensation (MC) and rate-distortion optimization (RDO).  Instead, the focus will be on a holistic approach where such functions are "machine learned" from real video data. Recently, a  number of difficult problems in computer vision  have been solved with machine learning based on deep convolution neural-networks (DCNN).  Given that video compression can also be viewed as a computer vision problem,  this research will pursue the novel idea of using DCNNs for MC and RDO . The second major difference of proposed research is the use of bio-inspired vision models for performance optimization. Examples are retinal model for quantifying the distortion between original and coded video and models for information coding in human eye. These models can be used to heavily compress video data with a negligible effect on the quality as perceived by a human viewer. This is in contrast to the current practice of  compressing video based on  pixel-level differences which results in disturbing visible artifacts in compressed video. It is anticipated that this research will introduce novel frameworks for MC and RDO  which can help develop new video compression algorithms achieving a very high bandwidth efficiency. Being mainly carried out by graduate students, this research will also play an important role in the training of highly qualified personnel (HQP) in communication engineering.