During a four-week R&D endeavour, two Product Designers, three Engineers and a Team Lead worked to create Channels, a mobile app. Channels looks at how mixed reality can allow for collaboration, community and creativity in the city.
Simple tools can allow for creativity and collaboration - allowing residents to add markup layers to urban spaces, providing unique insights, narratives, and perspectives. Channels can be a scaffold for the new digital milieus of our urban geographies.
Channels looks at how mixed reality applications can exist in such a way as to allow for collaboration and creativity in the city. The first iteration of Channels is a proof of concept of what will be possible in the near future when spatial mapping and Cloud AR allow for the relocalization of mixed reality scenes.
At its beginning, Channels was an exploration into critical urban issues. Toronto is currently facing a tidal wave of dense residential development, and our access to green and open spaces is rapidly dwindling. As a result, parks—and less traditional public spaces like the Bentway—have become all the more special and sought out by city dwellers.
Urban community gardens, children drawing with chalk, duelling chess players, and unique artistic events are ways in which people play in these public spaces—outside of the confines of their 400 square ft boxes in the sky. Our team saw an opportunity to make parks even more flexible, or malleable, to the different needs of all urban residents. As technologists, we saw an opportunity to create an even bigger sense of “space” for city dwellers with the flexibility offered through the virtual overlays of augmented reality.
Key Terminology
Augmented Reality (AR): Augmented Reality or AR experiences are when virtual objects are superimposed over a person’s experience of the real world. These objects can be visual, auditory, haptic, and even somatosensory and olfactory.
Spatial Maps: Spatial maps help devices to understand their physical environment so that they can realistically place virtual objects in that environment.
Localization: Localization technology allows devices to ground their location in an augmented reality scene by generating and comparing spatial maps through the combination of digital video and GPS sensors.
Persistence: Persistence refers to when virtual objects can remain in the same physical location in which they were previously positioned—so that they can be revisited, viewed, and interacted with after closing and re-opening an AR application.
Cloud Anchors: Cloud Anchors are hosted in the cloud and so allow augmented experiences to persist in the physical world by multiple users across space and time.
Benchmarking
Google’s Just a Line introduced a new way for people to play in AR. Just a Line is an app that lets users draw on the world with their friends. This creative spin got our team thinking: How might AR inspire people—and communities—to play in public spaces, like parks?
While we were thinking about communities, we started looking into the long history of digital community platforms like Internet Relay Chat (IRC) networks, internet forum-based communities like The Well, and more contemporary platforms like Slack. We felt that community and collaboration would be key in making an AR experience compelling for people.
Introducing Channels
Channels is an AR application for smartphones that helps people who want to explore and contribute to their local communities by providing them with co-creative tools that encourage collaboration.
The app works like this: users open the app when in a physical location, like a park. Various "channels" or groups are revealed to them within the app that are location-specific, like #grangeparkflowers, #racoonwatch or #tic-tac-toe. These are groups that anyone can create or join and contribute to.
To illustrate how Channels works, it’s useful to imagine how it would really work and wrap up that imagined scenario into a story. Lauren enters her local park and opens up her Channels app. There, she sees different communities she can join: one, where people are playing tic-tac-toe by drawing virtual Xs and Os in the sky. In another, people are leaving virtual sticky notes to indicate budding plant types. in yet another example, people have started to use virtual building blocks to design unique sculptural expressions. Lauren joins the tic-tac-toe community and unfinished grids that she can add to are revealed in the space around her.
We found inspiration for our UX flows by auditing Slack's channel creation and browsing flows.
This allows us to create a similar medium for organic growth and collaboration, Channels is an internet relay chat that is overlaid on our city, with unlimited layers.
New channel creation flow
Users can tap into these channels where they can view and collaboratively create and build with other users around the channel's topic, using tools like an AR paintbrush.
Channel Relocalization Flow
Once our team figured out what we wanted to create, we needed to work in a way that would allow the product designers to explore from an experiential and visual perspective, and the engineers to tackle the technical challenges involved in creating a new kind of mobile AR experience. We worked similarly to how we do most of our projects at Connected: an agile, dual-track process where design and technical learnings rapidly feed into one another. We set up daily and weekly cadences for our small team, with stand-ups, collaborative whiteboarding sessions, and prioritization meetings that allowed us to quickly come to a minimal viable product (MVP) vision to work towards.
Persistent Scenes with Cloud AR
While Just a Line provides users with the ability to create ephemeral drawings in AR, Channels does something new. Amongst the plethora of mobile AR apps hitting the market channels manages something entirely new. It allows users to create, revisit, and interact with their creations—and the creations of others—even after closing and reopening the app. This is called persistence and because it’s not formally supported by ARkit and ARcore it was brand new territory that we were exploring. Additionally, rather than every user experiencing the same park, with the same AR creations, users can create communities through what are called #channels.
Key Terminology
Creating our own persistent cloud anchors with AWS s3 and ARkit.
When we did this work a year and a half before persistent cloud anchors became available, however shared and persistent anchors is what we were needing. We did a technical audit of both ARcore & ARkit in order to understand if we could accomplish our objectives using either of them.
ARcore at the time had cloud anchors, but they weren't persistent. ARkit didn't provide cloud anchors, but it did allow us to manage our own spatial maps, which we could store in the cloud ourselves.
In order to create shared and persistent drawings, we would need to relocalization of scenes
Demo
To illustrate how Channels works, it’s useful to imagine how it would really work and wrap up that imagined scenario into a story. Lauren enters her local park and opens up her Channels app. There, she sees different communities she can join: one, where people are playing tic-tac-toe by drawing virtual Xs and Os in the sky. In another, people are leaving virtual sticky notes to indicate budding plant types. in yet another example, people have started to use virtual building blocks to design unique sculptural expressions. Lauren joins the tic-tac-toe community and unfinished grids that she can add to are revealed in the space around her.
Summary
Channels explores new ways of connecting people and fostering community within neighbourhoods. It acts as a proof of concept towards a speculative future where hyper-local culture is supported on a platform that mixes the physical environment with new digital realities.
Through a shared passion for urbanism and new technologies, our team explored the current realm of spatial computing and, in doing so, started to understand the potential futures it can unlock. As designers, technologists and product developers we believe that it is imperative that we think more critically about the societal, and ethical implications of the future we are helping to build. Our team has spent a lot of time discussing and debating the impacts of Channels, and other mixed reality technologies. These discussions have led us to insights that have gone far beyond our initial MVP vision for Channels, and into foresight work around spatial computing.