This project was proceeded by a previous research engagement, which you can read about here. Following our concept showcase presentation, our stakeholders had given us the green light to begin validating our two concepts through a make-to-learn project which consisted of validation testing and rapid and iterative prototyping. We would have 7 weeks to take these concepts and interrogate their value propositions through iterations of designing prototypes, testing with users, and incorporating our learnings. At the kickoff of this project we also restructured the team with two researcher strategists rolling off, and two engineers rolling on.

We then broke down our customer segmentation further to ensure that we were recruiting a diverse group of participants for our research. We wanted to ensure they'd have differing motivations and comfort levels when it comes to riding micromobility vehicles.

We used a hypothesis-driven validation methodology, which is a systematic approach to validating an idea by breaking it down into testable hypotheses in order to gradually reduce risk through evidence-based decision making.

To figure out where to start in our validation testing I began by identifying the assumptions we were making in our value proposition(s). These assumptions are what we need to be true in order for our value proposition to meet the needs of our users and be desirable to them.

I reframed these assumptions into hypotheses using We Believe That statements and then prioritized them by criticality and risk. This began with identifying any potential 'idea killers'–any assumptions that we would want to begin testing and validating immediately. These hypotheses might include risks of desirability, feasibility, usability or viability.

After identifying what we needed to validate I wanted to take some time to understand which approaches would help us learn the most, in the shortest amount of time. If we don't need to develop functional code-based prototypes, we shouldn't. At the same time, for some of our hypotheses, especially those around the spatial rendering of audio, we would need to ensure a level of experiential fidelity. Below are the types of prototypes that I evaluated in this process.

After this initial evaluation, and running a scoping session with our engineers, I selected two prototyping methods that would give us the flexibility to validate our assumptions and provide the highest degree of experiential fidelity.

The test setup was quite unique and complex so I'll take a moment to unpack how it was set up. The tests were conducted by a road test liaison in partnership with myself, acting as the remote operator. The liaison facilitated our tests with the participants, riding with them between waypoints in the city, and conducted a short interview with them between each test. As the remote operator, I stayed in our office, operating our WoZ prototypes using a tool we developed specifically for this project.

I was able to listen to our research participant and their interactions with our voice assistant using a microphone attached to their helmet. I could then trigger our voice assistant's responses with a web application we developed. The web application would communicate with the mobile application which would synthesize speech locally on a mobile device stowed away in a small backpack we had each participant wear during our road test. This synthesized speech and media would playback on a pair of smart glasses.

As the operator, I was able to listen to the participant's voice on a one-way call, while simultaneously on a two-way call with the liaison. To augment this line of communication the participant's location was being broadcast to me with real-time GPS tracking, providing me with the additional context that was essential in order to effectively operate a successful WoZ test.

In previous work, out of necessity, I'd already developed a WoZ design and testing tool, Simili, and initially, I considered using it to help us with our prototyping. I realized quickly however that we would need to develop new capabilities for it, and this might take too long. Alternatively, we could develop something entirely new. After scoping both options and for the sake of velocity we went with the latter and designed and built something extremely lightweight, and including only the functionality we needed.

I needed a testing tool that would help us to validate some of our assumptions regarding spatial audio, acoustic augmented reality, and location-aware conversational agents. I hypothesized that by combining the audio augmented reality SDK our client had developed with an open-source library I had found from Google called Resonance we would be able to do the real-time spatial audio rendering we needed. Our engineering pair was able to build a very small proof of concept that this would work to validate this approach was feasible and out of necessity, and we went forward and developed this prototyping software in less than two weeks, which required a lot of quick problem solving and prioritization to accomplish.

Instead of designing the entire front end of a WYSIWYG (what you see is what you get) design tool, I discovered that we might be able to just use Google's My Maps as a design canvas. This was ideal because I had already been using it to plot our test segments and design our prototypes to be location context-aware. I discovered that from My Maps I could export a KMZ file, which I could then convert to a CSV file. This allowed me to design and export files for the dialogue and import them into our own testing software. The CSV files contain the dialogue for our conversational agent as well as geolocation coordinates, media file names, and directional angle values for spatial rendering.

The operator imports into the operator's test console a CSV file that loads the preconfigured prototype design, visible as a 'list of sounds' as well as anchored within a Google Map. Each prototype can consist of dozens of geolocated sounds and conversational prompts. A connection is then established between the web client and the companion Android application on a device carried by the test participant. The Android application is also sending the device's GPS coordinates to the operator's client and their location is updated in real-time for the operator to monitor within the test console.

The operator can toggle between an automatic or manual mode. In automatic mode, the audio prompts will trigger when the cyclist enters a tight radius to the geolocated prompt. In manual mode, however, the operator is responsible for triggering each prompt, as in a traditional WoZ test. In order to make manual mode easier, prompts that are nearby the test participant are dynamically suggested to the operator, making finding the correct ones quicker. If necessary, the operator can also improvise and type responses, which proved invaluable when our participants made unexpected turns, or when we wanted to reflect real-world conditions, such as the rider's speed or the weather.

In addition to text string for speech synthesis, the triggered prompts can include the names of audio files, as well as angle degrees from true north. Since there are no fields for angle degree or attaching audio files in the design tool–Google My Maps, I established a text formatting in the description field that I was already using for text-to-speech dialogue. To distinguish these values we used predefined special characters which could then be parsed by our companion Android application.

In the title field, I would use a dollar symbol proceeded by a value representing the degree angle from true north that they want the angle to play from. I'd also indicate the playback of a sound file using $sound in the description field followed by the filename. All of the earcons or music files I used in the prototypes were stored on the device and can be accessed by the companion Android app when needed for spatial rendering.

To create spatial audio the AR software development kit provides the ability for us to connect the Android application with real-time positional data from a pair of Smart Glasses, and we use the Google Resonance library to reference that data and render all our audio spatially at the desired angle. Multiple sounds can be overlaid at different angles. This allows us to playback earcons (sound UI) and synthesize speech on a 360-degree soundstage. We found contrasting angles make the 360 soundstage more evident. However, sometimes normal stereo is more suitable. When no angle is selected, the sound is synthesized as head locked stereo (standard stereo).

We ran this test with 11 participants in 5 different scenarios, which included prototypes that I'd designed with side-by-side comparisons with Google Maps’ turn-by-turn directions with our turn-by-turn directions enhanced by spatial audio cues. We also tested 'tour guide' features that took advantage of spatial audio causes to provide branching navigation and points of interest throughout the city.

In order to increase the experiential fidelity of this prototype, I incorporated a stationary bike that our participants would ride while viewing the VR prototype. Using screencasting the facilitator was able to view whatever the participant was seeing on a nearby monitor.

Below is a demo of one of our tour guide prototypes. Youtube supports VR180 video and first-order ambisonic audio so If you wear headphones while viewing this video you'll be able to hear the ambisonic spatial audio. Use your mouse to simulate your head movement and you'll hear the audio pan accordingly to keep the soundstage in the correct orientation.

In this tour guide scenario, the rider experiences a guided tour of Trillium Park. Points of interest are highlighted during the tour and the user is able to ask questions about the park in an intuitive way because the assistant has the directional context of the user's head position. This allows the assistant to understand "What kind of trees are those", and understand which ones the user is looking at.

To help us measure the effectiveness of spatial audio in enhancing navigational prompts and lowering cognitive loads for riders. We tested multiple variants of an experience where the voice assistant was providing the user with turn-by-turn navigational prompting, and several versions which used spatial audio in different ways, and contrasted this to our control.

I designed our VR prototypes not only to validate that spatial audio had value for micromobility riders but also where and how spatial audio can best be applied. This resulted in a set of applied principles that help users more intuitively distinguish and interpret spatial audio cues.

We faced a lot of uncertainty when it came to hardware, as a result, we tested several formats, systematically switching throughout our road tests while documenting user feedback.

With each participant, we conducted a 2-hour session where they were taken through several scenarios during road tests on bike share and electric scooters. This allowed us to validate and invalidate our various hypotheses about our concepts hardware and software features and in a real-world setting. Each session included a half-hour context interview, followed by a 1.5 hrs road test that included four different prototypes, between each prototype was a five-minute experience review interview.

While our focus was on the core features of navigation and safety we also explored some additional use cases and learned that across the wide range of riders, meeting some of these lower priority jobs can create motivating gain creators. Fitness tracking, music and tourism all provide interesting avenues to create additional value for some customer segments.

Throughout our testing, we also gained valuable insights into hardware. Insights that helped highlight potential feasibility risks, and enabled us to make evidence-based recommendations on potential integrations with our client's existing product line, in addition to the HMI opportunity that the rise of shared bike and scooter systems were presenting.

Our client selected us to continue this project further into hardware development and a functional code-based prototype. An additional and surprising outcome of this early prototyping and experimentation was the identification and filings for two patents.

Our product owner had recognized there was a wealth of interesting concepts, some of which were tangential to our work but interesting nonetheless, so he provided us with a form to submit concepts. Two ideas that I had submitted were selected by an innovation board as being novel and patent-worthy. Subsequently, after this phase of the work, I was asked to collaborate with a team of technical writers and a legal team to develop the ideas further, document them, and file for two US patents. To give myself time to do this I was moved off the project, but continued to act as an advisor during the next phase of work that involved the development of a functional prototype, an multimodal, voice-first HMI for micromobility users.