Grant: Discovery Grants Program - Individual
University of Victoria
$28,000.00
Time Information for Multimedia Processing and Analysis
FY 2014-15
Summary
The amount of multimedia data available digitally is growing daily at an astounding rate. One of the main challenges in multimedia research is to develop algorithms to extract information from these vast collections of multimedia signals and provide more effective tools for interacting with them. Using a variety of techniques from digital signal processing and machine learning it is now possible to analyze millions of images, videos and songs. Another recent trend is the use of rich expressive ways of interacting with computers moving beyond the traditional screen/keyboard/mouse paradigm. These include gestures on touchscreens as well as full body tracking interfaces based on a variety of sensors such as accelerometers, microphone arrays and depth cameras using structured light. There are few examples of work that combine large scale analysis of multimedia archives controlled by rich multi-modal input. Interactive music performances that combine sensors, actuators, computers and humans are one of the few areas where such synergies are taking place. Building such systems is challenging as not only it requires interdisciplinary expertise but the majority of existing programming environments address only some of the issues that arise. They also lack generic abstractions for expressing time and probabilities which are fundamental in building such interactive systems. Our main objective is to formalize, design and implement better tools, languages and frameworks for building interactive multimedia tools that involve both large-scale machine learning and rich user input. Modeling time is fundamental to interactive multimedia and our goal is to explore how time can become more explicit and part of the abstractions used to program. We plan to elevate concepts from statistics, machine learning, signal processing and data-flow programming from arbitrary conventions in a software library to first class primitives in programming practice.
Program
Discovery Grants Program - Individual
Recipient
Location
British Columbia, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data
As published
Application id
298270-2012
Name nom
Tzanetakis, George
Department département
Computer Science
Organization id
7
Institution établissement
University of Victoria
Province en
British Columbia
Province fr
Colombie-Britannique
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2014
Competition year année de concours
2012
Award amount
28000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
802
Area of application group en
Information and communication services
Area of application group fr
Services d'information et de télécommunications
Area of application en
Information systems and technology
Area of application fr
Systèmes d'information et technologie de l'information
Research subject code
2721
Research subject group en
Information technology
Research subject group fr
Technologie de l'information
Research subject en
Multimedia systems and networks
Research subject fr
Systèmes et réseaux multimédias
Partie
3
Nb partie
5
Application title
Time Information for Multimedia Processing and Analysis
Keyword
NSERC
Application summary
The amount of multimedia data available digitally is growing daily at an astounding rate. One of the main challenges in multimedia research is to develop algorithms to extract information from these vast collections of multimedia signals and provide more effective tools for interacting with them. Using a variety of techniques from digital signal processing and machine learning it is now possible to analyze millions of images, videos and songs. Another recent trend is the use of rich expressive ways of interacting with computers moving beyond the traditional screen/keyboard/mouse paradigm. These include gestures on touchscreens as well as full body tracking interfaces based on a variety of sensors such as accelerometers, microphone arrays and depth cameras using structured light. There are few examples of work that combine large scale analysis of multimedia archives controlled by rich multi-modal input. Interactive music performances that combine sensors, actuators, computers and humans are one of the few areas where such synergies are taking place. Building such systems is challenging as not only it requires interdisciplinary expertise but the majority of existing programming environments address only some of the issues that arise. They also lack generic abstractions for expressing time and probabilities which are fundamental in building such interactive systems. Our main objective is to formalize, design and implement better tools, languages and frameworks for building interactive multimedia tools that involve both large-scale machine learning and rich user input. Modeling time is fundamental to interactive multimedia and our goal is to explore how time can become more explicit and part of the abstractions used to program. We plan to elevate concepts from statistics, machine learning, signal processing and data-flow programming from arbitrary conventions in a software library to first class primitives in programming practice.