Canada Spends Logo

Explore our Budget 2026 analysis

Grant: Discovery Grants Program - Individual

York University

$28,000.00

New computational models of human visual perception of surface colour, 3D shape, and lighting

FY 2023-24

Summary
What could be easier than seeing? We see without trying, and usually without thinking about it. Seeing seems easy because our visual cortex is a powerful computing device with a lifetime of experience. One reason why vision is a challenging computational task, though, is that all images are highly ambiguous: any given image could conceivably be seen as depicting a wide range of shapes, colours, and lighting conditions. In order to perceive things correctly, our visual system must overcome this ambiguity. The long-term goal of my research program is to develop computational models that solve this problem in the same way people do, and see what people see in complex, realistic scenes. The research proposed here will approach this goal in two ways. First, using powerful new methods developed for machine learning and computer vision, I will train artificial neural networks to perceive surface colour, 3D shape, and lighting conditions in complex scenes. To do this I will use rendering software to generate training data that includes images of many diverse scenes, along with representations that show the true colour and shape of objects in those scenes, as well as the true lighting conditions. The networks will be trained to view just the images, and deduce the shape, colour, and lighting in the scenes as well as possible. Previous research suggests that simply because the networks are trained on naturalistic data, they will have some of the same characteristics as human vision: they will find the same visual tasks easy or hard, and they will see many of the same illusions. I will test this prediction, and as it will probably not turn out to be completely true, I will also revise the networks as necessary so that they see shape, colour, and lighting as similarly to human vision as possible. The second approach I will take is to run perceptual experiments with human participants that investigate what fundamental visual features make up our visual world. For example, we obviously perceive colour and 3D shape, and just as obviously we do not perceive the polarization of light. I will focus these experiments on the claim that 'brightness' is a fundamental perceptual dimension, defined as the point-by-point intensity of images (technically, 'perceived luminance'). I will systematically vary surface colour and lighting of test patches in real and computer-generated scenes, and measure how these variations affect judgements of surface colour and brightness. These measurements will help to establish whether 'brightness' is a feature that we actually perceive, separate from surface colour and lighting conditions. This research will help us to understand normal human vision, both in real life and in the simulated virtual environments that are becoming increasingly important for many applications. It will also provide information that will be useful for developing computer vision systems that see what people see in complex, realistic scenes.
Program
Discovery Grants Program - Individual
Location
Ontario, Canada
Recipient Type
Academia
Type
Grant
Source
NSERC's Awards Data

As published

Application id
RGPIN-2022-04583
Name nom
Murray, Richard
Department département
Psychology
Organization id
38
Institution établissement
York University
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2023
Competition year année de concours
2022
Award amount
28000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1502
Committee name en
Biological Systems and Functions
Committee name fr
Systèmes et fonctions biologiques
Area of application code
1210
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Psychology
Area of application fr
Psychologie
Research subject code
9999
Research subject group en
Not available
Research subject group fr
Not available
Research subject en
Non disponible
Research subject fr
Non disponible
Application title
New computational models of human visual perception of surface colour, 3D shape, and lighting
Keyword
3D shape perception; colour constancy; computational modelling; deep learning; human visual perception; intrinsic images; lighting perception; lightness perception; machine learning; psychophysics
Application summary
What could be easier than seeing? We see without trying, and usually without thinking about it. Seeing seems easy because our visual cortex is a powerful computing device with a lifetime of experience. One reason why vision is a challenging computational task, though, is that all images are highly ambiguous: any given image could conceivably be seen as depicting a wide range of shapes, colours, and lighting conditions. In order to perceive things correctly, our visual system must overcome this ambiguity. The long-term goal of my research program is to develop computational models that solve this problem in the same way people do, and see what people see in complex, realistic scenes. The research proposed here will approach this goal in two ways. First, using powerful new methods developed for machine learning and computer vision, I will train artificial neural networks to perceive surface colour, 3D shape, and lighting conditions in complex scenes. To do this I will use rendering software to generate training data that includes images of many diverse scenes, along with representations that show the true colour and shape of objects in those scenes, as well as the true lighting conditions. The networks will be trained to view just the images, and deduce the shape, colour, and lighting in the scenes as well as possible. Previous research suggests that simply because the networks are trained on naturalistic data, they will have some of the same characteristics as human vision: they will find the same visual tasks easy or hard, and they will see many of the same illusions. I will test this prediction, and as it will probably not turn out to be completely true, I will also revise the networks as necessary so that they see shape, colour, and lighting as similarly to human vision as possible. The second approach I will take is to run perceptual experiments with human participants that investigate what fundamental visual features make up our visual world. For example, we obviously perceive colour and 3D shape, and just as obviously we do not perceive the polarization of light. I will focus these experiments on the claim that 'brightness' is a fundamental perceptual dimension, defined as the point-by-point intensity of images (technically, 'perceived luminance'). I will systematically vary surface colour and lighting of test patches in real and computer-generated scenes, and measure how these variations affect judgements of surface colour and brightness. These measurements will help to establish whether 'brightness' is a feature that we actually perceive, separate from surface colour and lighting conditions. This research will help us to understand normal human vision, both in real life and in the simulated virtual environments that are becoming increasingly important for many applications. It will also provide information that will be useful for developing computer vision systems that see what people see in complex, realistic scenes.
York University - $28,000 Grant (2023-24) | Canada Spends