Grant: Discovery Grants Program - Individual

Imagine a visual world of random blotches of colour and texture, like an abstract painting. This is a world without shape, and it illustrates how our capacity to see structure and recognize objects is determined by our ability to perceive shape. In the proposed project we will employ a novel combination of psychophysical and computational methods to determine how the human brain extracts and represents 2D and 3D shape information from contours in complex imagery, and will develop improved computer vision algorithms for object segmentation and shape processing based upon these insights.*The seemingly effortless way in which we perceive shape belies a daunting complexity. As you gaze around you now your visual world is likely a complex clutter of partially occluded objects, variegated lighting and shadows. These complexities fragment objects into perceptual shards that the brain must correctly group together in order to compute accurate representations of shape. This process of perceptual organization is a combinatorial problem of exponential complexity, yet the brain solves it reliably and efficiently, vastly outperforming current computer vision algorithms. This impressive performance appears to derive from the brain's ability to fuse multiple local and global grouping cues within a recurrent hierarchical neural architecture. A major deliverable of the proposed work is a detailed and testable computational model of this neural circuit informed by a key set of new psychophysical experiments.*Natural shapes generally sweep out low-dimensional curved manifolds in a high dimensional shape space. While most effort in computer vision has focused on discriminative methods for separating these manifolds, a fully generative model is required to support perceptual grouping and many other tasks. The challenge is thus to identify efficient but fully generative models of shape. Prior work has been challenged by topological instability, but our recent formlet theory of shape solves this problem. A second deliverable of the proposed work is a fully probabilistic graphical formlet model of 2D shape, evaluated psychophysically as a model for human shape representation.*While most studies of 3D shape perception focus on depth cues to shape such as stereopsis, or surface cues such as shading, the 2D shape of the bounding image contour is also known to strongly influence 3D shape perception. A third deliverable of the project is a new probabilistic model for estimation of 3D shape from the bounding contour, represented using a 3D extension of our formlet theory.*These results will substantially advance our understanding of shape processing in human visual cortex. The algorithms developed here will form the basis for applications in automatic video analytics for traffic surveillance and for 2D to 3D film conversion.