Grant: Discovery Grants Program - Individual

The ability to move our bodies with skill and purpose is easily taken for granted as humans. However, understanding and replicating such capabilities is an open problem. In computer graphics and simulated worlds, human motions are often the least-realistic aspect of the virtual worlds being portrayed. In the real world, the sensorimotor capabilities of robots remain greatly impoverished as compared to humans and animals.  Recently, significant strides forward have been made using deep reinforcement learning (DRL), albeit with significant caveats related to the results, i.e., often simulation-only, low visual quality, inefficient learning, and very tightly circumscribed motion tasks. What methods and advances are needed to endow physically-simulated characters and physical robots with the movement patterns and motor skills that are comparable to, if not better than, those seen in humans and animals? What is the right languages and tools for effective authoring and design of new skills? The objective of my research program is to answer these questions, with the help of suitable representations, learning algorithms, and data. Structured RL for Movement: A key open problem is that of finding or learning good building blocks that support efficient learning of movement-related tasks. I will develop and evaluate methods that: embed knowlege of body structure and kinematics into the structure of policy networks; leverage learned abstractions; explore learning in different policy structures that support sequential and parallel (in time) composition; and learn to put a new dynamical system into correspondence with a canonical dynamical system that we already know how to control. Flexible Autoregressive Models: Human motion capture data can be used to learn models that are predictive of future movement. Using large collections of motion data, we can learn a model of all motion possibilities that are available at any given point in time. Using a class of variational auto-encoder model, we can then use reinforcement learning, a type of trial-and-error learning, to learn how to automatically produce realistic sequences of movement that achieve given tasks. Modular Sim-to-Real Testbed: How can new learning capabilities for controllers be used to consider hardware designs that previously would have been impractical because of limited control capabilities? How can efficient learning be achieved when the data comes from a set of individually-unique robots? We aim to advance the state of the art in robot-controller co-design, as well as to test for "sim-to-real" generalization, i.e.,  the transfer of results from simulation onto real robots. We will leverage the expertise of my research group in deep reinforcement learning and sim-to-real, and extend this towards soft robotics and modular robotics. We will collaborate with experts in modular robot design and computational fabrication in order to realize the hardware components.