Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Discovery Grants Program - Individual

University of Alberta

$18,000.00

Perceptually Enhanced Multimedia Data Acquisition, Processing, Transmission and Visualization

FY 2015-16

Summary
Incorporating human perceptual criteria along the processing pipeline from data acquisition to visualization can deliver better visual experience to users. Despite the latest advances in high definition technology, the dynamic range supported by digital display devices is still insufficient to match the physical range of luminance in natural scenes as perceived by the Human Visual System. Photometric equipment can only capture a subset of this natural spectrum. Thus, the challenge lies in encoding a scene where light variation spans beyond this subset. This can be partly addressed by acquiring multiple images at different exposures, fused into one final High Dynamic Range (HDR) image, which then undergoes a mapping process and is output to Low Dynamic Range displays for human viewers. Current HDR scene acquisition methods rely on fusing images of a scene captured at different exposures. This approach does not work for dynamic scenes. I propose exploiting a multi-view acquisition technique to capture a scene in high dynamic range using cameras with different exposures. The cameras need to be accurately calibrated and the goal is to recover under- or over-exposed details in one image from other images by probabilistically interpolating neighbourhood pixels based on local and global characteristics. My new technique will have the added advantage of enhancing the fused content using depth cue. The perception of depth has been studied intensively and shown to be effective in providing good viewing experience for humans. Despite the existence of many state-of-the-art multi-exposure fusion methods, what has not yet been considered is a computational model that further optimizes the luminosity, contrast and color saturation of fused pixels based on their relative depths in the scene - a topic to be studied in my research. My new technique can be applied to images, videos, and 3D models. The research theme of this proposal has four major components: (1) Acquisition: Effective HDR and depth data acquisition; (2) Processing: A novel Saliency-Depth-Contrast-Saturation computational metric for enhancing image quality; (3) Transmission: Efficient coding and compression integrating perceptual parameters in the proposed metric along with analytics of visual data, including dynamic mesh and human motion animation; and (4) Visualization: Objective and subjective validation of a proposed metric addressing the cross-display and retargeting scenarios. The significance of my work lies in improving the perceptual quality of multimedia acquisition, processing, transmission and visualization. New perceptual parameters, including depth and saliency, will enhance traditional measures. The outcomes will be compared with state-of-the-art methods. My techniques will help design future high definition displays to deliver user desired quality of experience.
Program
Discovery Grants Program - Individual
Location
Alberta, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
RGPIN-2015-04232
Name nom
Cheng, Irene
Department département
Computing Science - Computing Science
Organization id
9
Institution établissement
University of Alberta
Province en
Alberta
Province fr
Alberta
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2015
Competition year année de concours
2015
Award amount
18000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
1207
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Information, computer and communication technologies
Area of application fr
Technologie de l'information, des ordinateurs et des réseaux de communications
Research subject code
2708
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Image processing
Research subject fr
Traitement des images
Partie
1
Nb partie
1
Application title
Perceptually Enhanced Multimedia Data Acquisition, Processing, Transmission and Visualization
Keyword
NSERC
Application summary
Incorporating human perceptual criteria along the processing pipeline from data acquisition to visualization can deliver better visual experience to users. Despite the latest advances in high definition technology, the dynamic range supported by digital display devices is still insufficient to match the physical range of luminance in natural scenes as perceived by the Human Visual System. Photometric equipment can only capture a subset of this natural spectrum. Thus, the challenge lies in encoding a scene where light variation spans beyond this subset. This can be partly addressed by acquiring multiple images at different exposures, fused into one final High Dynamic Range (HDR) image, which then undergoes a mapping process and is output to Low Dynamic Range displays for human viewers.<br /> <br /> Current HDR scene acquisition methods rely on fusing images of a scene captured at different exposures. This approach does not work for dynamic scenes. I propose exploiting a multi-view acquisition technique to capture a scene in high dynamic range using cameras with different exposures. The cameras need to be accurately calibrated and the goal is to recover under- or over-exposed details in one image from other images by probabilistically interpolating neighbourhood pixels based on local and global characteristics. My new technique will have the added advantage of enhancing the fused content using depth cue. The perception of depth has been studied intensively and shown to be effective in providing good viewing experience for humans. Despite the existence of many state-of-the-art multi-exposure fusion methods, what has not yet been considered is a computational model that further optimizes the luminosity, contrast and color saturation of fused pixels based on their relative depths in the scene - a topic to be studied in my research. My new technique can be applied to images, videos, and 3D models.<br /> <br /> The research theme of this proposal has four major components: (1) Acquisition: Effective HDR and depth data acquisition; (2) Processing: A novel Saliency-Depth-Contrast-Saturation computational metric for enhancing image quality; (3) Transmission: Efficient coding and compression integrating perceptual parameters in the proposed metric along with analytics of visual data, including dynamic mesh and human motion animation; and (4) Visualization: Objective and subjective validation of a proposed metric addressing the cross-display and retargeting scenarios. <br /> <br /> The significance of my work lies in improving the perceptual quality of multimedia acquisition, processing, transmission and visualization. New perceptual parameters, including depth and saliency, will enhance traditional measures. The outcomes will be compared with state-of-the-art methods. My techniques will help design future high definition displays to deliver user desired quality of experience.