Designing AR for technicians

Exploring emerging tech for mobile workers




Retriever Communications


My role

Note: I’m covering only a portion of the project to avoid sharing proprietary information.

We explored possibilities for emerging technologies to benefit the work practices of field technicians. The focus of our work was a case study of lift technicians in Sydney’s CBD, working together with an established lift service provider. Our project included a participatory design workshop, three site visits, and two rounds of design work and evaluation.

We focused on how technicians perform work and communicate, and how current technologies support or hold them back. From there, we created a small number of design concepts and prototypes focused on mobile technologies and augmented reality. These were intended to spark discussion, identify opportunities for innovation and identify design criteria that will help ensure new developments have a positive impact on the performance and experiences of field technicians.

Of the three concepts that were developed, here I’ll only cover a data visualisation application that shows the real-time position of an elevator on three different devices: an augmented reality headset (Microsoft Hololens), a smartphone, and a smartwatch.

Manual brake lifting scenario

Following the initial workshop and site visits, we sketched out several interesting scenarios. For each scenario we ideated in which ways emerging technology, such as augmented reality, could deliver value. For example, occassionaly a technician has to manually move a lift into a safe position so stranded passengers can disembark. The technician would accomplish this by lifting the lift motor brakes slightly. However, because this is such a rare last-resort approach, there is not always a clear feedback mechanism to ensure the lift is brought to a halt at exactly the right height. Sometimes, the control equipment that shows whether a lift is in a safe position so its doors can open is positioned just out of sight.

Sketching out the above scenario: In very rare circumstances, when people get stuck in a lift, a technician has to manually lift the engaged brakes in the motor room to move the lift car to a nearby floor so that stranded passengers can get off safely.

Ideation and concept development

Initial sketches for working out options to visualise lift position in augmented reality. One of the ideas considered was to project the lift's position onto the mechanical structure. However, with different layouts, and considering the position of the technician relative to the equipment, that wouldn't work consistently.

Mock-up of the interface for a phone, made in Figma. At this stage, we sought further feedback on our concept directions from our industry partner.

Prototype apps on phone and Hololens

All three apps (on a smartwatch, Hololens AR headset, and shown here, on a phone) were built as functional prototypes by Andrew Bluff. An iPad app was used as a server, simulating a lift and its data responses.

Point-of-view screenshot of the Hololens prototype app. The idea is that a technician can pin the 3D lift shaft graphic to any position in their surroundings with voice commands, leaving their hands free to operate equipment.

Video of the lift position application in augmented reality on a v1 Hololens, starring Andrew Bluff and some HVAC equipment as stand-in for a lift motor.

Partial findings

In our (confidential) report, we discussed a number of recommendations for the use of AR in industrial contexts. The three most relevant to the lift position example are:

  1. General – Despite developments in work practices, the core of working on a lift may not shift much over time since this equipment remains operational and mostly as-is for decades. Thus, novice technicians need to familiarise themselves with equipment and technical interfaces from decades past. AR, or even smartphone interfaces, could add a consistent data overlay minimising differences in lifts' technological eras.
  2. Specific to app – Use audio to augment (or replace) visual displays in situations where technicians require unimpeded vision for other tasks: For example, a technician may be busy adjusting equipment or be required to peer into hard-to-see spaces. To remain aware of systems’ status, carefully designed sonifications of that data can be used. These may be ‘sonar beeps’ to indicate distance, bell chimes to indicate correct adjustment of a particular parameter or even spoken voice announcements.
  3. Specific to app – Provide direct and consistent control over visuals and sounds to ensure workers are able to manage their cognitive load: Because augmented reality claims a portion of someone’s field of view, it is necessary to ensure workers are not overloaded with information and remain aware of their surroundings. This means that devices must ensure that data displays, alerts, etc., can be controlled and, where necessary, minimised or turned off quickly.