Grant: Discovery Grants Program - Individual
The University of British Columbia
$66,000.00
Frameless video system: emulating the human visual system
EF 2023-24
Résumé
The real world is analog and continuous in the most practical sense. The sound we hear has an amplitude variation over time, in a continuous and analog fashion. The light we sense through our eyes is also continuous both in amplitude and time, as the eye's retina receives a continuous flow of photons, sending that information to the brain through a chemical reaction. However, conventional video cameras cannot capture light information the way the human sees it, as they are limited to capturing discrete frames over time. Movies and videos are clever and sophisticated inventions that give the illusion of motion and realism to the human eyes, yet fooling them to believe they are seeing a faithful replica of the real world. The challenge of capturing video in discrete frames goes back to 1892, when Edison and Dickson developed a device called the Kinetoscope to view the resulting motion picture by playing back about 15 frames per second. It was until later that the addition of sound forced the frames to increase to 24 per second. Recently, these rates have significantly increased, but we continue treating video as a discrete sequence of images, imposing an unnecessary constraint on pixel information, dynamic range and precision, causing motion artifacts, compression challenges and an overall limited quality of experience. More importantly, this stroboscopic viewing is definitely tiresome to humans, as the eye and the brain have to provide the temporal integration necessary to perceive continuous motion. We believe that, considering the recent advances in video capturing and processing technologies, the time is ripe for a paradigm shift in the way we capture, process and display visual information, closely emulating the human visual system and its ability to perceive the continuity of the real world. Our long-term objective is to embark on a better understanding of the continuous time video representation and develop the framework and necessary technologies that will address visual quality, storage, delivery, and display requirements for frameless video, thus arriving at a more accurate rendition of the real world that is authentic and more pleasing to the eye. The implication to the digital media world is very significant. If frameless video shows promise, the broadcasting and entertainment industry, as well as the consumer electronics industry will have to embrace the opportunity to migrate to new ways to deliver entertainment, news and social media content to consumers. We suspect a similar reaction will have to occur for the computer vision community, driving significant new market opportunities across the digital media industries which represent some of the fastest growing sectors in the world, with a market growth rate of 25% and global sales forecast to hit US$215 billion by 2021.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
British Columbia, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2019-05551
Name nom
Nasiopoulos, Panos
Department département
Electrical and Computer Engineering - Electrical and Computer Engineering
Organization id
2
Institution établissement
The University of British Columbia
Province en
British Columbia
Province fr
Colombie-Britannique
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2019
Award amount
66000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1510
Committee name en
Electrical and Computer Engineering
Committee name fr
Génie électrique et informatique
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2514
Research subject group en
Electrical and electronic engineering
Research subject group fr
Digital signal processing
Research subject en
Génie électrique et électronique
Research subject fr
Traitement des signaux numériques
Application title
Frameless video system: emulating the human visual system
Keyword
Capturing; Compression; Frameless video; Signal processing; Visual system
Application summary
The real world is analog and continuous in the most practical sense. The sound we hear has an amplitude variation over time, in a continuous and analog fashion. The light we sense through our eyes is also continuous both in amplitude and time, as the eye's retina receives a continuous flow of photons, sending that information to the brain through a chemical reaction. However, conventional video cameras cannot capture light information the way the human sees it, as they are limited to capturing discrete frames over time. Movies and videos are clever and sophisticated inventions that give the illusion of motion and realism to the human eyes, yet fooling them to believe they are seeing a faithful replica of the real world. The challenge of capturing video in discrete frames goes back to 1892, when Edison and Dickson developed a device called the Kinetoscope to view the resulting motion picture by playing back about 15 frames per second. It was until later that the addition of sound forced the frames to increase to 24 per second. Recently, these rates have significantly increased, but we continue treating video as a discrete sequence of images, imposing an unnecessary constraint on pixel information, dynamic range and precision, causing motion artifacts, compression challenges and an overall limited quality of experience. More importantly, this stroboscopic viewing is definitely tiresome to humans, as the eye and the brain have to provide the temporal integration necessary to perceive continuous motion. We believe that, considering the recent advances in video capturing and processing technologies, the time is ripe for a paradigm shift in the way we capture, process and display visual information, closely emulating the human visual system and its ability to perceive the continuity of the real world. Our long-term objective is to embark on a better understanding of the continuous time video representation and develop the framework and necessary technologies that will address visual quality, storage, delivery, and display requirements for frameless video, thus arriving at a more accurate rendition of the real world that is authentic and more pleasing to the eye. The implication to the digital media world is very significant. If frameless video shows promise, the broadcasting and entertainment industry, as well as the consumer electronics industry will have to embrace the opportunity to migrate to new ways to deliver entertainment, news and social media content to consumers. We suspect a similar reaction will have to occur for the computer vision community, driving significant new market opportunities across the digital media industries which represent some of the fastest growing sectors in the world, with a market growth rate of 25% and global sales forecast to hit US$215 billion by 2021.