Grant: Collaborative Research and Development Grants

Using tracking technology and pervasive video capture technology, it is possible to automatically tag salient objects in the video space. These semantic tags provide rich video content that can be used to provide users with new types of video experiences including context and content sensitive non-linear video browsing, advanced video search, and personalized videos that can be shared, saved and searched. For one example we consider instrumenting a hockey arena with video cameras and tracking technology. The video data streams to a video server continuously and is processed to locate players in the various video feeds. As players are tracked we detect when they are in view of each video camera and estimate the quality of view on a per player or group of players basis. Then, using a special video browser called a Diver, users can select the player they would like to watch and camera angles will automatically switch to maintain the best view of that player. Video summaries may also be created through queries to the video space to provide sequences of customized content. Semantic tags also allow us to support video hyperlinking, which provides connected sets of video content such as player statistics, advertising, coaching tips and so forth. Thus, combining a video processing back-end (to create semantically tagged video content) with a set of video interfaces provides new forms of video experiences. We are investigating new algorithms for tracking and recognizing people in videos and creating a scalable, user-centric vision language that will allow these algorithms to be embedded in fast processing architectures. From this infrastructure we are developing two applications that utilize this rich data set. The first uses the computer vision back-end to create a personal video organizer to assist people with managing their own videos. The video collection will be searchable and contains links to extended content about their personal videos such as names, places and other videos. The second application is a new video browsing environment where users' video experiences are recorded so that they can be searched, organized and shared. We intend both applications to be used in both desktop and mobile environments.