Grant: Alliance Grants

Volumetric media (VM) such as volumetric video, mesh or point-cloud streaming technologies are the future of fully immersive VR/AR/XR experiences, which are becoming increasingly popular and are being used in a variety of applications (e.g., remote medicine, metaverse, and gaming). The challenges surrounding VM technologies are tremendous due to the high network bandwidth needed to produce high-quality and low-latency streams. Although, some studies have investigated aspects to mitigate these challenges, but mostly in isolation, and without considering holistically the end-to-end full transport workflow, especially in the context of a novel, mobile networks such as 5G. Designing an end-to-end VM transport with optimized quality of experience (QoE) is still a relatively new research topic and is challenging. This challenge is also confirmed by engineers from our industry partner, InterDigital, a pioneering R&D company in mobile/video technologies. The main goal of the proposed project is to explore and address key challenges in the transport of VM using QUIC, and to identify the best practices for using QUIC effectively in different scenarios, given the new network realities. In particular, we aim to address the following interrelated research tasks (T1-T6) holistically together: how to efficiently (T1) package, define and implement necessary QUIC-based (T2) payload formats and messages needed for signaling VM encoded using Volumetric Video-based Coding to allow (T3) flexible and robust quality adaptations and viewport predictions under extremely (T4) varying network conditions and with (T5) very low latency. We also aim to (T6) define and implement a mechanism on top of the proposed system to evaluate the performance of other transport protocols such as WebRTC and HTTP adaptive streaming. The outcome of the project will significantly improve the R&D activities for VM in InterDigital as well as contribute to Canada's VM industry competitive advantage, and globally, for the years to come. It also offers the exciting opportunity to train 5 HQP who will develop expertise in Networked Multimedia Systems and Immersive Media Technologies, which are competencies highly demanded in the job markets.