Grant: Discovery Grants Program - Individual

The dream of useful robots, such as companion robots for the elderly or autonomous automobiles, requires major advances in the computational realization of visual cognition. I and my trainees have previously presented theory, algorithms and systems that successfully demonstrate a wide variety of the component capabilities of such visual cognition, but we have not yet put the pieces together into a whole functioning system. An old but influential concept, the visual routine suggested by S. Ullman in 1984 seems to present the right foundation for this integration. Ullman suggested that a process he called a visual routine, composed of sequences of elemental operations could be responsible for computing qualities such as shape or spatial relations. Such qualities are important for visual cognition in general. Many have looked at this idea in the past but with limited success. I feel the idea continues to have merit and that perhaps the lack of progress is due to the fact that no one has yet updated the visual routines idea given a current understanding of vision and visual attention; a vast amount has changed in our understanding since 1984. The proposed research will re-visit the visual routines idea in the context of a current understanding of vision and visual attention, focussing on how our past work may be extended to define a new conceptualization of visual routines.The role of visual perception, for both humans and robots, is not to simply categorize single images. It is to enable an understanding of the physical world and to guide action and manipulation within that world. Without the ability to visually reason about what is seen, this goal cannot be realized. The proposed research will contribute to this goal and will be tested on my lab's visually-guided robots. Not only would advances in this area bring closer the dream of companion robots for the elderly or autonomous automobiles, but they would also contribute to more useful visual surveillance systems or to robots that may safely interact with humans in manufacturing settings, among other applications.