Seeing the World through Digital Prisms: Psychological Implications of Passthrough Video Usage in Mixed Reality

Authors: Jeremy N. Bailenson, Brian Beams, James Brown, Cyan DeVeaux, Eugy Han, Anna C. M. Queiroz, Rabindra Ratan, Monique Santoso, Tara Srirangarajan, Yujie Tao, and Portia Wang

Abstract: Millions of people will soon be spending hours each day relying on cameras and screens to show them the surrounding world. Apple, Meta, and other companies are mass-producing headsets that block out light from the real world and instead rely on passthrough video as an enabling technology for mixed reality. The 11 authors of this article each spent a number of hours wearing these headsets in public and in private, with the goal of documenting experiences in passthrough to then organize and review previous research that will help research scholars, industry leaders, and other organizations better understand psychological consequences over time. First, we describe why passthrough will become an essential component of the media landscape. Next, we summarize the technological specifications that make new passthrough headsets stand out from previous ones, but still have lower fidelity compared to human vision on parameters such as field of view, distortion, latency, and resolution. Next, we review relevant previous psychological research. We conclude that the passthrough experience can inspire awe and lend itself to many applications but will also likely cause visual aftereffects, lapses in judgments of distance, induce simulator sickness, and interfere with social connection. We recommend caution and restraint for companies lobbying for daily use of these headsets and urge scholars to rigorously and longitudinally study this phenomenon.

Link to paper