Grant: Research Tools and Instruments - Category 1 (<$150,000)

We propose to purchase a high-quality indoor motion-capture system to track the locations of a team of flying robots and humans as they interact, cameras for data collection by a team of robots, and computer servers and storage to process these data. This equipment will support the applicants' individual and joint research programs. Each of our team has recently demonstrated world-leading new capabilities in multi-camera tracking and SLAM (Tan), in vision-mediated human-robot interaction (Vaughan), and human activity recognition from video (Mori). The proposed equipment will support our new joint work, where we (i) control a team of flying robots to observe moving people from several viewpoints (ii) reconstruct a 3D view of each person and extract a skeleton pose estimate in real time, (iii) categorize the people's activity, and (iv) use the activity information to help the robots improve their camera placement.The key piece of proposed equipment is an indoor motion capture system to provide ground-truth measurements that allow us to analyze and evaluate the performance of our system. When proven indoors, our goal is to have the robot system work outdoors. Multi-view stereo and 3D modeling of people and the environment is a rapidly-growing area with great commercial promise. The new ubiquity of camera-equipped mobile devices means that our algorithms could be used immediately by hundreds of millions of people. The use of robots as automated camera platform is just starting commercially, and will be very valuable. Finally, accurately capturing the pose of a freely-moving person has major applications in health diagnosis, entertainment, and human-robot interfaces.