Grant: Idea to Innovation

We are aiming to develop and evaluate prototypes of a head-free 3D display system in anticipation of transferring our technology to e-commerce and video-based communication. Visual depth serves a number of different functions and is communicated by a number of different visual qualities. Standard screens of smart phones, tablets and regular computers convey only pictorial depth cues, such as occlusion and linear perspective. Some more sophisticated screens are set up for stereoscopic depth, but generally require special headgear to secure delivery of different images to the two eyes. Stereopsis then conveys information about distance but is still not able to mend the lack of directionality experienced when looking at screens. The lack of directionality becomes particularly obvious in screen-based telecommunication. If person A looks into his camera, person B feels being looked at no matter where he is located in front of her screen. If person A is not looking into the camera, her gaze seems to be offset from B's perspective. The offset remains even if B tries to catch A's gaze by moving into its line. Lacking directionality means that an important communication channel is missing, which is likely a main contributor to "Zoom fatigue". Directionality requires a very specific visual quality: motion parallax, which implies that the direction from which a visual scene is seen changes as the observer moves. Our technology implements motion parallax by tracking the user's head location in front of a screen with device-mounted sensors and updates the view presented on the screen accordingly. In three phases, we first apply motion parallax to visual exploration of passive objects, such as merchandise presented in an online shop (MPdepth I). In the second phase, we implement a visual telecommunication system where the two participants are presented by live-animated computer graphic avatars (MPdepth II). In the third phase eventually, we replace computer graphics contents with 3D video: Multicamera recordings of the person are streamed as input for a viewpoint interpolation