Transforming airline operations with AR and VR

Neil Redding ,

Robert Drotar and

Scott Dickson

Published: July 18, 2017

Wilbur Wright was airborne for 12 seconds when he piloted his first powered aircraft flight. He covered 120 feet above a windy beach in North Carolina. The descent cracked the skid under the rudder, but the landing was otherwise smooth. How far we’ve come! Aircraft today are sophisticated digital and mechanical systems. Despite leaps and bounds in progress, they still break. And that’s hardly a surprise. Planes cart passengers to all corners of the earth in a complex world of extreme heat, cold and turbulence.

Maintenance technicians are a critical element of the airline operations process. Their job keeps passengers and planes safe. Speedy and accurate repairs mean reduced flight delays and more in-service, revenue-generating time for aircraft. Drawn out repairs mean longer delays and big financial penalties for airlines. Boeing estimates that, depending on the operator and age of an aircraft, an aircraft on the ground for one or two hours can cost anywhere between $10,000 and $150,000. Virtual and Augmented Reality can help technicians get their work done faster, with a higher degree of safety. And it’s not just for technicians - VR and AR can improve training effectiveness and enhance job performance for everyone from flight attendants to ground staff.

Transforming enterprise workflows

Technicians across industries work in challenging environments. These aren’t your average desk jobs. Unlike their office-dwelling counterparts, it’s difficult for technicians to pull up reference materials in a work context. No nearby desks to leave manuals on or monitors to keep endless tabs open in a hangar or in the field. In fact, technicians might have to walk hundreds of meters to consult a physical or digital document or look up an aircraft’s maintenance history.

They’re required to perform an incredibly wide range of tasks on many different kinds of aircraft. Most often, they might perform routine repairs like fixing reading lights or replacing a worn out tire. Less often, a more involved repair might demand that they troubleshoot a complex avionics system like a heads-up display. Certain complex repairs might only need to be performed very infrequently.

Now: VR for training

VR will transform training for airline staff. An immersive experience can give a new technician a multidimensional look at a complex problem that would be hard to duplicate in a typical training environment. They could perform a virtual repair and gain the experience needed for work in the field. A more experienced technician could also use a VR simulation for recurring or refresher training before tackling a procedure they have not done recently, or at all. VR is a no-risk way of learning new skills without risk of damage to actual components.

An instructor can be in one place and the technicians can be distributed across a wide geography. Yet, they can gather together in the same virtual workspace to review a specific training scenario. This reduces an airline's need to pull mechanics off the line and send them to a training classroom in another city. Training quality improves and the airline saves costs and time away from work for the technician. The same approach can be applied to other staff, too. Flight attendants could learn how to operate new equipment and deliver redesigned onboard service. Ground staff could learn the proper way to maneuver ground service equipment and sequence the complex array of servicing tasks during an aircraft turn. Immersive VR training can save hundreds of millions of dollars of annual damage repairs and delay costs.

Next: The augmented technician

The physical environment and complex nature of the work provides a compelling use case for AR to transform the technician’s workflow. A wearable or mobile device could layer digital content into the technician’s work environment - for example presenting them the tasks that need to be done for the day or suggesting the most efficient way to perform a repair. They could easily access the plane’s work history or summon other relevant documents via voice command. How might this work? We created a proof of concept to demonstrate this scenario.

AR can also allow on site operators to summon a technical expert from anywhere in the world. The technician could augment their existing skills, digitally marking up the safest and quickest way to complete a repair. Immersive, real-time assistance has the potential to significantly reduce downtime and time spent looking for the right information. Remote assistance also provides a solution for the shortage of fully qualified, trained and certified maintenance personnel.

As regulations evolve, integrating AR support with the airline’s ERP or MRO solution might also allow for quick and easy reporting via recorded video, reducing liability and the burden of administration, and allowing the technician to spend more time in the field.

The future of aircraft maintenance

AR is one of a number of emerging technologies that will transform the way airlines perform maintenance, repair and overhaul. IoT, Conversational UI, AI and Machine Learning (ML) all will play increasing roles in improving the accessibility of parts and information, as well as better informing and interactively guiding staff as they collaborate and perform their respective tasks.

New aircraft are outfitted with sensors and recorders that collect vastly more and more granular data. Using this data, AI/ML are already starting to predict when maintenance and repair tasks will need to be done, as well as providing real-time assistance as these tasks are performed. The future will see humans working in close collaboration with machines - connected hardware and intelligent software - to achieve better outcomes in more efficient ways.

Beyond aircraft maintenance

And aircraft maintenance is just one of a large set of operational scenarios that will be transformed by these technologies. Similar scenarios like repair, maintenance, and training for rail and road vehicles, energy production and distribution, pipelines for oil/gas/water/etc distribution, medical scenarios, to name a few.

How soon will AR become mainstream as a tool to transform the way workers across industries perform their jobs? Now that it’s clear Apple will bring mobile AR to the close to 1B iOS devices in use by autumn of this year, we expect enterprise applications of AR to expand from the early adopters to widespread use in the not too distant future.

Now is the time to get started

As with compounding interest, it pays big to get started with these technologies sooner. The companies that will reap the competitive advantages of this transformation are the ones investing in it today.

Disclaimer: The statements and opinions expressed in this article are those of the author(s) and do not necessarily reflect the positions of Thoughtworks.

Services

Industries

Resource Hubs

Publications and Tools

All Insights