Augmented Reality: An MRO On-The-Job Training Force Multiplier
By Marlo Brooke
Aircraft maintainers constantly tackle complicated tasks under tight deadlines and regulations. Because of the demands that are the MRO community’s fact of life, most training processes are a combination of On-the-Job Training (OJT) and 2-D computer and paper reference manuals. OJT is very effective. It’s broadly used in almost every organization because it works. Sort of.
Because OJT also comes at a very high price. Pulling a senior maintainer to train a junior technician not only almost doubles the hours of the procedure being repaired OJT (combined trainee and trainer time), it also stops the senior maintainer from performing their own repairs. This leads to more backlog, overtime, and delays. These hidden costs and time delays degrade both productivity and profit.
Technology will never replace a human being’s knowledge – especially when it comes to highly complex MRO tasks. The key is to make what works even more effective, with the use of technology. This means better supporting the maintainers who are doing this important, highly regulated, safety-of-life critical work. Enter Augmented Reality.
Augmented Reality (AR) for business began to gain realistic traction in 2014. Primarily funded by innovation funding by the government, many years were spent making AR good enough to support enterprise production operations.
Fast forward to 2022, and most organizations are planning and implementing a digital transformation strategy that includes a roadmap for AR. How does this all play into the MRO gameplan?
Technology sometimes overpromises and underdelivers. When we were approached 8 years ago by the US Navy to help support aircraft availability (given that Readiness was a major concern due to rising political concerns), we asked the obvious question: “What is your measure of success when it comes to aircraft availability?”
Their answer: “We want a system for our maintainers that will help them maintain faster with zero errors.”
Back then, AR was not enterprise ready, let alone something remotely usable by the US Department of Defense. But with much effort and industry advancements, we found a way to help them hit their success metric: Maintain faster with zero errors.
Today MRO can take advantage of these early research efforts that have proven out the technology. AR is a proven OJT enabler that significantly reduces the cost and burden associated with traditional OJT. AR can be thought of as a “Virtual Mentor” for the technician. Utilizing a comfortable AR-enabled wearable device usable in a maintenance setting, the technician sees relevant contextual information overlaid on the equipment when, where and how he/she needs that information to be effective. This frees senior mentors to impart their knowledge — less so on the what/how of the task, and more on the why behind the operation.
An OJT Force Multiplier Use Case
In 2019 we developed an AR application for a complex engine repair procedure reserved for only the most experienced senior level maintainers. During the user acceptance testing, a novice technician with no previous maintenance experience was able to complete a complex maintenance repair procedure successfully using ARrepair – the first time around – with minimal mentor guidance. The novice, brand new at the job, had never even seen the engine prior to doing this procedure. Yet with ARrepair on a wearable HoloLens, he was able to concurrently learn each step (guided in 3D reality) while on the job, do, check, and proceed successfully, in an acceptable amount of time. As you might imagine, this proven use case changes the operational model of maintenance.
ARrepair would never have been successful without the accomplished experience of the senior maintainer, who acted as a subject matter expert to ensure the AR procedures followed precise instructions. AR essentially captures the most important MRO knowledge acquired over decades, and delivers that knowledge when, where, and how the technician needs it. With this step-by-step linear procedure, the technician no longer needs to consult a 2-D diagram or PDF instruction to gain and interpret information. They are guided with 3-D visual and audio cues, just as if they had someone with them and talking and guiding them through the entire procedure.
ARrepair connects to sensor data from equipment devices (Internet of Things), enabling the maintainer to see real-time status information within the wearable device. On certain systems this can also be interactive, meaning the maintainer can change sensor settings or throw digital switches without ever touching the equipment.
Artificial Intelligence fuels automatic quality checks in ARrepair. Just as a good mentor carefully watches the trainee technician and has him or her go back and fix any small error, ARrepair uses AI to track the trainee’s actions (including eye tracking and intricate hand movements) and guides the technician to go back and fix something. The application can then automatically inspects (QA) the equipment itself to ensure correct repair.
With performance metrics captured by the AR software, a senior level maintainer can now focus his or her efforts on working with the technician to fine-tune both of their efforts.
AR with AI is a powerful On the Job Training force multiplier that makes the new MRO technician more effective and efficient while both learning and doing the repair. AR also empowers the mentor to precisely impart their knowledge and ensure that the job is getting done right, the first time around, with zero errors.