View-Dependent Video Textures for 360° Video

Authors
Sean J. Liu, Maneesh Agrawala, Stephen DiVerdi, Aaron Hertzmann

Image: In 360◦ video, viewers can look anywhere at any time. In the opening scene of Invasion!, a rabbit emerges from a cave (a). In sequential playback, a viewer looking at the cave (green box) will see the rabbit emerge, whereas a viewer not looking at the cave (red box) will miss this event. We provide tools to guarantee that viewers see the region of interest (ROI) at the correct timecode to witness the event (b). We introduce the concept of gated clips, where playback only continues if the viewer satisfies a condition related to the ROI (green boxes). Otherwise, our player loops the video using view-dependent video textures (red boxes).

Abstract
A major concern for filmmakers creating 360° video is ensuring that the viewer does not miss important narrative elements because they are looking in the wrong direction. This paper introduces gated clips which do not play the video past a gate time until a filmmaker-defined viewer gaze condition is met, such as looking at a specific region of interest (ROI). Until the condition is met, we seamlessly loop video playback using view-dependent video textures, a new variant of standard video textures that adapt the looping behavior to the portion of the scene that is within the viewer’s field of view. We use our desktop GUI to edit live action and computer animated 360° videos. In a user study with casual viewers, participants prefer our looping videos over the standard versions and are able to successfully see all of the looping videos’ ROIs without fear of missing important narrative content.

The research was published in Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology on 10/23/2019. The research is supported by the Brown Institute Magic Grant for the project Neverending 360.

Access the paper: http://dx.doi.org/10.1145/3332165.3347887
To contact the authors, please send a message to lsean@cs.stanford.edu