Learning from Project Channels
Sandbox was an ongoing R&D project to expand on some of our learnings from our previous mobile AR experiment–Channels. We had significant UX challenges with Channels that held back the true potential of collaboration and creation in shared AR spaces.
- AR drawing is hard for users on mobile AR platforms (vs headset AR/VR). It’s hard to view your sketch through the screen you are also drawing with.
- Relocalizing AR scenes is a challenge without a global spatial map. It's quite difficult to get users to the precise location needed to use SLAM to relocalize an AR scene.
The Opportunity
We saw an opportunity to build on the learnings of work on Channels. Specifically by improving the creative tools available to users, and more intuitive scene discovery and relocalization mechanics. We gave ourselves these two provocations.
How Might We Let mobile AR users find AR experiences by increasing discoverability and reducing the friction involved in relocalizing AR scenes?
How Might We connect mobile AR users through play and encourage collaboration in public spaces?
Encouraging Creative Play
We realized that the drawing tool in Project Channels was proving difficult to use and was not promoting the kind of play and collaboration we hypothesized it would. I researched deeper into the history of creative and collaborative play to understand it better.
How Might We connect mobile AR users through play and encourage collaboration in public spaces?
I had realized during Project Channels user testing that the 3D drawing tool was proving difficult for the majority of people, it was difficult to use effectively and drawings were challenging to interpret. This friction was not promoting the kind of play and collaboration we assumed it would.
In this next experiment, I identified two API's that would provide us with 'blocks' to play with, rather than lines. 3D objects provided by Google Poly, as well as animated .gifs provided by GIPHY.
Promoting Discovery & Enabling Relocalization
QR codes and hashtags are already widely used to connect physical space to digital space, over the last ten years they've both become broadly understood interaction patterns. We hypothesized that a similar interaction could help mobile AR users to discover and relocalize AR experiences.
How Might We connect mobile AR users through play and encourage collaboration in public spaces?
Benchmarking
When I looked for comparative products I found an example from the Google Lens team that demonstrated the use of a marker and label that indicates to users that an AR experience is available.
Introducing Sandbox
Sandbox is a platform for kids (and adults!) to discover and locate invisible AR play areas, where they are able to collaboratively co-create and play together through a set of creative tools.
Sandbox can connect us through the invisible AR spaces that surround us
A mobile application that unlocks the potential of collaborative co-creation in persistent and shared augmented reality place spaces.
Our concept envisions a near future where signage, floor tiles and printed materials are used by brands to indicate where XR (mixed reality) experiences are located nearby.
It’s a simple solution that creates awareness in the physical environment and solves a UX problem that exists in the near term as the public gets accustomed to new spatial computing technologies.
Signage
In order to promote discovery and for our application relocalize AR scenes from the cloud our concept includes signage with an AR marker. Throughout our office we placed signs at gather places to denote areas designated for AR play. A problem arose in our design and development process when ARcore's image anchor service became unreliable, this happened everytime we have more than a handful of signs. Signs in our database would get confused for one another and we noticed we were it was returning very low confidence scores. To improve performance we experimented with several variations of signage and measured them for performance.
We discovered through experimentation that although our signs had different emojis or patterned frames they were too similar, we were considering building a custom solution when we stumbled on a solution that was simple and effective, albeit unintuitive. When photos or artwork was set as the entire background, with full bleed, we were able to get consistent and reliable matches from the ARcore image anchor service.
User Journey
8 different locations were selected throughout the office to test Sandbox. Signs in each location show the location itself as well as AR objects within them, as well as instructions on how to download the application.
Discovery
Scan & unlock scene
Contribute and create scenes
Collaborate on a scene
Multiple users can log into the same Sandbox scene and create and edit it collaboratively
Asynchronous collaboration
We wanted Sandbox to support the same kind of asynchronous collaboration that we had explored in our previous research with Channels. Both of the challenges we had faced with Channels regarding discovery and scene relocalization were being addressed by our signage system. In this demo you can see a sign is discovered, and the scene is unlocked. We introduced a new animation over the sign to signal a successful scanning. The holiday scene is unlocked and the user decides to add to the scene with three penguins models. This will automatically update the scene on our database, allowing it to be discovered by anyone in the future who can build upon it.
Synchronous Collaboration
As well as wanted to create an experience that enabled people to build together in a collaborative way. To accomplish this we established some basic logic around how often the application pushes updates to the s3 storage, and experimented with it iteratively to avoid editing conflicts between users. We consider this type of multiuser interaction pattern as 'Google docs for AR'.
