Grant: Strategic Projects - Group

Aiming at real-time simulation and interaction via multi-sensory modalities such as vision and touch, virtual reality (VR) finds many applications including oil and gas exploration. However, adaptation of VR applications has been slow, due to a bottle-neck issue related to intuitive user interaction. This issue becomes more problematic when VR applications run across a computer network for multi-user interaction. We propose to create a hands-free interactive system for manipulating object-oriented information within networked virtual environments. We will develop self-contained desktop VRs and interconnect them to form a networked virtual environment. Each desktop VR will generate 3D stereoscopic displays of the Reservoir Simulation - a software product from the Computer Modelling Group Ltd (CMG). Underpinning the desktop VR will be a hands-free interactive system. This interactive system will utilize camera-based gesture recognition (extracting the user intention of interacting with objects of the 3D displays) and haptics (simulating the sense of touch by providing force feedback to the user's hand). The framework of integrating both gesture recognition and haptics will employ the neural mechanisms that humans use to perform visual perception and visually guided actions. The interactive system will allow multiple users to collaborate their individual interaction (via each desktop VR) across the networked virtual environment. The novelty of the project lies in the integration of gesture recognition and haptics for user interaction; the use of new 3D range cameras for gesture recognition; and the application of the neural mechanisms to address engineering issues. Our previous work confirms that it is achievable to create the hands-free interactive system using current technologies and knowledge. As a novel tool for reservoir simulation, the hands-free interactive system will propel effective oil-related services and economic development for Canada. Its benefit will be allowing the CMG to lead a niche market of reservoir simulation. Its impact will go beyond the oil-related services, serving as a bridge to connect and enhance innovative VR-based technologies for other applications.