We wanted to explore the user experience of one of the concepts we had developed in our Looking Glass project. We called it 'XR Window' at the time and we quickly explored the UX of this concept with a 2-week sprint where we developed a code-based prototype using ARkit. Using head tracking, wide angle video, and virtual cameras can we want to create a stronger sense of co-presence between two spaces, in essence–a window.

Using head tracking camera vision to extrapolate approximate x,y,z axis position of user allows for the reprojection of the virtual camera and the creation of the illusion of looking through a window. The camera hardware on Facebooks portal devices are extremely wide angle and this allows for a 'look through' parallax when using a screen as a window frame. This illusion is particularly impactful when looking into a virtual scene in VR space but also may be a compelling UX for the Portal to Portal calls.

In 2 weeks we designed and built our prototype that leverages the illusion of perspective to give a higher degree of presence to its users than standard video conferencing. Given the very short nature of this experiment, and our recent technical exploration of ARkit on iOS we decided to explore the UX design rather than the feasibility of head tracking that would be required to develop this solution further.

In development, we tried a variety of scenes, including video game scenes such as this one. Portal windows like this could potentially serve as a window in the Metaverse, or portals between Metaverses.

The AR experience in this video is actually only on the iPhone, but this allowed us to quickly understand and communicate the value of future development.

Our experiment helped us validate the potential product impact and communicate the concept but we do still see potential risks that would need to be understood better. In our literature review we identified that this effect often suffers from its 'flatness' and while it records well, the in-person the parallax effect can lack depth, and this hampers the success of the illusion. A potential solution maybe found in hardware by using a glasses-free 3d solution such as a lenticular lens screens. This experience also may transfer to VR and is another avenue to explore.