Grant: Discovery Grants Program - Individual
University of Toronto
$94,000.00
Computational and neural mechanisms of perceptual grouping
EF 2024-25
Résumé
How does the human brain sort visual information into meaningful units? And how can we use the insights from human vision to boost machine vision? My trainees and I will seek to answer these questions with a three-pronged approach: 1. develop computational algorithms for perceptual grouping in real-world images and test their utility for human vision; 2. measure their implementation in the human brain; 3. add perceptual grouping mechanisms to neural networks for computer vision to improve their performance and robustness.
Following extraction of simple visual elements early on, so-called mid-level human vision groups the elements into shapes that form the basis for high-level vision tasks: the recognition of objects, faces, and scenes. Between well-understood early vision and, thanks to fMRI, heavily researched high-level vision, the computational and neural mechanisms of mid-level vision, largely comprising perceptual grouping, are a glaring gap in our understanding of visual perception of real-world environments. With the proposed research my trainees and I seek to fill this gap to allow us to more fully grasp the cascade of computations performed by the human brain to derive meaning from the activations of photoreceptors in the retina. We will develop computational algorithms for perceptual grouping in real-world images, test their utility for human vision, and explore their implementation in the human brain. The work will transform our understanding of human vision by moving perceptual grouping from the realm of a seemingly random set of qualitative rules into the realm of concrete computational and neural mechanisms.
Artificially intelligent systems have started to permeate our daily lives - think of self-driving cars, refrigerators that recognize their own content, or cell phone cameras that are able to frame the best shot for a photo fully automatically. These systems need to deal with a cluttered, real-world environment. However, despite their impressive recognition performance, computer vision algorithms based on convolutional neural networks (CNNs) are fooled by adversarial images as well as stimuli as simple as silhouettes and glass figurines. We argue that this is due to a lack of mid-level vision, leading to CNNs learning texture statistics rather than organizing edge elements into shapes. We will use the insights from human vision research to implement perceptual grouping rules for CNNs. Applied to computer vision, perceptual grouping will endow artificially intelligent agents with more robust vision systems for comprehending the real-life situations, in which they are increasingly deployed. Such a mechanistic understanding of perceptual grouping is likely to give rise to advances in computer vision that will allow AI systems to better interact with the messiness of the real world that they cohabit with us.
Programme
Discovery Grants Program - Individual
Bénéficiaire
Emplacement
Ontario, Canada
Type de bénéficiaire
Academia
Type
Grant
Source
NSERC's Awards Data
Tel que publié
Application id
RGPIN-2020-04097
Name nom
Bernhardt-Walther, Dirk
Department département
Psychology (St. George Campus) - Psychology (St. George Campus)
Organization id
31
Institution établissement
University of Toronto
Province en
Ontario
Province fr
Ontario
Country en
CANADA
Country fr
CANADA
Fiscal year exercice financier
2024
Competition year année de concours
2020
Award amount
94000
Program id
RGPIN
Program name en
Discovery Grants Program - Individual
Program name fr
Programme de subventions à la découverte - individuelles
Group en
DISCOVERY RESEARCH
Group fr
RECHERCHE AXÉE SUR LA DÉCOUVERTE
Committee code
1502
Committee name en
Biological Systems and Functions
Committee name fr
Systèmes et fonctions biologiques
Area of application code
1210
Area of application group en
Advancement of knowledge
Area of application group fr
Avancement des connaissances
Area of application en
Psychology
Area of application fr
Psychologie
Research subject code
6301
Research subject group en
Psychology
Research subject group fr
Psychologie
Research subject en
Sensory systems and perception
Research subject fr
Systèmes sensoriels et perception
Field of research list names en
Not available
Field of research list names fr
Non disponible
Socioeconomic objective list names en
Not available
Socioeconomic objective list names fr
Non disponible
Application title
Computational and neural mechanisms of perceptual grouping
Keywords
perceptual grouping; mid-level vision; scene perception; visual perception; fmri; psychophysics; gestalt psychology; symmetry; parallelism; convolutional neural networks
Application summary
How does the human brain sort visual information into meaningful units? And how can we use the insights from human vision to boost machine vision? My trainees and I will seek to answer these questions with a three-pronged approach: 1. develop computational algorithms for perceptual grouping in real-world images and test their utility for human vision; 2. measure their implementation in the human brain; 3. add perceptual grouping mechanisms to neural networks for computer vision to improve their performance and robustness.
Following extraction of simple visual elements early on, so-called mid-level human vision groups the elements into shapes that form the basis for high-level vision tasks: the recognition of objects, faces, and scenes. Between well-understood early vision and, thanks to fMRI, heavily researched high-level vision, the computational and neural mechanisms of mid-level vision, largely comprising perceptual grouping, are a glaring gap in our understanding of visual perception of real-world environments. With the proposed research my trainees and I seek to fill this gap to allow us to more fully grasp the cascade of computations performed by the human brain to derive meaning from the activations of photoreceptors in the retina. We will develop computational algorithms for perceptual grouping in real-world images, test their utility for human vision, and explore their implementation in the human brain. The work will transform our understanding of human vision by moving perceptual grouping from the realm of a seemingly random set of qualitative rules into the realm of concrete computational and neural mechanisms.
Artificially intelligent systems have started to permeate our daily lives - think of self-driving cars, refrigerators that recognize their own content, or cell phone cameras that are able to frame the best shot for a photo fully automatically. These systems need to deal with a cluttered, real-world environment. However, despite their impressive recognition performance, computer vision algorithms based on convolutional neural networks (CNNs) are fooled by adversarial images as well as stimuli as simple as silhouettes and glass figurines. We argue that this is due to a lack of mid-level vision, leading to CNNs learning texture statistics rather than organizing edge elements into shapes. We will use the insights from human vision research to implement perceptual grouping rules for CNNs. Applied to computer vision, perceptual grouping will endow artificially intelligent agents with more robust vision systems for comprehending the real-life situations, in which they are increasingly deployed. Such a mechanistic understanding of perceptual grouping is likely to give rise to advances in computer vision that will allow AI systems to better interact with the messiness of the real world that they cohabit with us.