Grant: Alliance Grants

This research project aims to transform video gaming by introducing advanced interactivity through technology that converts speech into expressive, real-time facial animations. This innovation will enable game characters to exhibit fluid, lifelike emotions in response to player actions and choices, leading to games that are both interactive and adaptive, offering a seamless, immersive experience. This project tackles two main challenges: first, accelerating the creation of detailed 3D facial models; and second, developing a method to accurately align these models with a character's speech in any language, accommodating a wide variety of face types, expressions, emotions, and speech patterns. These advancements will allow game characters to display a full range of emotions naturally and fluidly. Imagine a game where characters can express joy, surprise, or anger, all in response to the player's voice tone or choice of words, enhancing gaming with a deeply immersive experience where each interaction is unique and personal. Ubisoft, a leader in game development, is collaborating to integrate these innovations directly into new games, aiming to revolutionize interactive storytelling and game design.