Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Discovery Grants Program - Individual

University of Toronto

$73,000.00

Looking at People and Web-Scale Image Analysis

FY 2017-18

Summary
Humans are intensely social, and as such we spend a lot of time looking at people. With a simple glance we perceive pose, actions, interactions, intentions and emotions, all of which are necessary for effective social interaction. Future computers are going to spend as much time looking at people as we do. This will be important for the design of new man-machine interfaces, perceptive consumer devices, surveillance systems, aids for the visually impaired, and myriad systems we have yet to envision. This proposal aims to develop state-of-the-art models of human shape, appearance, kinematics and dynamics to facilitate video-based analysis of people. This entails new techniques for estimating human pose, motion, and interactions with objects in the environment. The core technologies have myriad possible applications, including those in health care, assessing recovery of locomotion after hip replacement, to assessing the efficiency of movement in high performance sport. One key to looking at people, and to vision more generally, is learning. Vision relies on sophisticated models of the visual world (e.g., object shape, appearance, motion, etc.), which comprise our prior beliefs that support our ability to make plausible inferences when the image alone only partially constrains the scene (which is true most of the time). And these models are so hard to specify from first principles, that it is often more effective to learn them from data. Doing so entails working with massive image or video corpora, a.k.a. big data. To facilitate storage and analysis of huge image corpora, we are developing new data structures and algorithms for fast modeling, indexing, search and retrieval of high dimensional data. We recently developed several key technologies that allow us to store billions of images on a single computer, and perform similarity search (finding database items similar to a query image) in a few milliseconds. We plan to continue improving and generalizing such techniques, allowing one to store, retrieve, model and learn (classification and regression) from massive datasets. This will facilitate improved learning in vision, and it will find applications in web-scale search and retrieval applications, like searching all the images on the web based on image content, but using a small fraction of the computational resources that are currently required by companies like Google to perform the same tasks.
Program
Discovery Grants Program - Individual
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
RGPIN-2015-05630
Name nom
Fleet, David
Department département
Computer and Mathematical Sciences (Scarborough) - Computer and Mathematical Sci
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2017
Competition year année de concours
2015
Award amount
73000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
Discovery grants programs
Group fr
Programmes de subventions à la découverte
Committee code
1507
Committee name en
Computer Science
Committee name fr
Sciences informatiques
Area of application code
1207
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Information, computer and communication technologies
Area of application fr
Technologie de l'information, des ordinateurs et des réseaux de communications
Research subject code
2603
Research subject group en
Robotics
Research subject group fr
Robotique
Research subject en
Computer vision
Research subject fr
Vision artificielle
Partie
3
Nb partie
5
Application title
Looking at People and Web-Scale Image Analysis
Keyword
NSERC
Application summary
Humans are intensely social, and as such we spend a lot of time looking at people. With a simple glance we perceive pose, actions, interactions, intentions and emotions, all of which are necessary for effective social interaction. Future computers are going to spend as much time looking at people as we do. This will be important for the design of new man-machine interfaces, perceptive consumer devices, surveillance systems, aids for the visually impaired, and myriad systems we have yet to envision. This proposal aims to develop state-of-the-art models of human shape, appearance, kinematics and dynamics to facilitate video-based analysis of people. This entails new techniques for estimating human pose, motion, and interactions with objects in the environment. The core technologies have myriad possible applications, including those in health care, assessing recovery of locomotion after hip replacement, to assessing the efficiency of movement in high performance sport. One key to looking at people, and to vision more generally, is learning. Vision relies on sophisticated models of the visual world (e.g., object shape, appearance, motion, etc.), which comprise our prior beliefs that support our ability to make plausible inferences when the image alone only partially constrains the scene (which is true most of the time). And these models are so hard to specify from first principles, that it is often more effective to learn them from data. Doing so entails working with massive image or video corpora, a.k.a. big data. To facilitate storage and analysis of huge image corpora, we are developing new data structures and algorithms for fast modeling, indexing, search and retrieval of high dimensional data. We recently developed several key technologies that allow us to store billions of images on a single computer, and perform similarity search (finding database items similar to a query image) in a few milliseconds. We plan to continue improving and generalizing such techniques, allowing one to store, retrieve, model and learn (classification and regression) from massive datasets. This will facilitate improved learning in vision, and it will find applications in web-scale search and retrieval applications, like searching all the images on the web based on image content, but using a small fraction of the computational resources that are currently required by companies like Google to perform the same tasks.