An Augmented Room Layout
The HoloLens team had just built an advanced technology for the Hololens 2, in summer of 2018 that could recognize and understand different surfaces like walls, floors, and ceilings. My task was to use this technology and demonstrate how onboard the for the app users could be improved. The prototype helped people quickly set up the app by automatically detecting the room's layout and improved setup speeds by over 200%
My Roles:
Team Lead
Research
Experience design
Implementation Strategy
Project Context
Scene Understanding:
We thoroughly tested and analyzed the API for the technology, and then focused on understanding how most people interacted with it. We created three versions of the API for a usability test:
Users tapped to define the observed label.
Users gazed at the object to label the surface.
Users gazed and gave a voice command.
We gathered 15 individuals with varying degrees of experience using HoloLens technology and had them try the different ways of setting up room space with labels in different orders. We ensured that there was no learning curve by changing the order of the experiment for each person.
The test showed that users needed real-time feedback to visualize the room scanning process. Users also preferred the version where they could air-tap to confirm the setup of a wall.
Layout Application
The App was tested in various environments to determine the best use of spatial identification technology. The on-boarding design involved three steps:
tapping the floor,
tapping the ceiling,
and tapping points on the wall.
However, this process quickly became tiresome and repetitive in larger rooms, causing hand fatigue. Our primary customer using this application were Factory floor setups, so the average rooms set up would take more like 45 minutes
Tech understanding
To start with, I wanted to learn the time and information gathering quality of Scene understanding a manual setup in Layout of the same office space in 45 seconds. As you can see the Image an top is where the SU technology captures a lot more and i was only able to manually capture a wall and the floor in the same amount of time.
Comparative research
Next I focused on also understanding the competitive landscape of parallel technologies built by ARKit, ARCore and Magic Leap
Room setup
The final design involved three steps: gazing at the floor and ceiling for automatic detection and labeling, using a walk-and-gaze approach for scanning walls and adding them with voice commands, and saying "done" to complete the process. Additional steps were included for editing and deleting walls.
Outcome
In testing, the setup time difference between the two experiences was only around 30 seconds in a small conference room. However, in larger spaces like auditoriums or customer locations such as factories and warehouses, the setup time was reduced by 250%. The ease of use and efficiency of the gaze and setup process, aided by Scene Understanding, allowed users to quickly walk around and perform just a few taps before the room was ready to be set up. The successful outcome of the project led to a live HoloLens demo presented at a company-wide event.