by Al Bredenberg, Senior Writer
Augmented reality (AR) is virtual reality’s lesser known twin. Rather than create a reality from scratch, as virtual reality does, augmented reality overlays digital additions onto real objects, enhancing what the viewer can see. Rapidly improving augmented reality technologies are giving the auto repair technician what amounts to “x-ray vision,” overlaying illustrations, videos, data, and instructions onto the physical car the technician is repairing. What’s more, with the recent addition of head-mounted smartglasses, the technician can now access content without the need to stop and pick up a tablet or smartphone (not to mention eliminating the need to leaf through a printed repair manual.) What’s more, connected-technology platforms can put shop technicians in touch with tech-support experts who can walk them through new or complicated repair procedures without interrupting the workflow.
BMW Group is convinced that augmented reality technologies are a worthwhile addition to the auto repair shop. The automaker recently rolled out AR smartglasses to all of its 347 service locations in the U.S. market, along with selected Mini dealerships. Announcements from Porsche Cars North America say that the company is deploying a similar system, called Tech Live Look, to its 189 dealerships in the U.S.
Will AR Transform Auto Service?
According to market-data firm Statista, the worldwide market for AR technologies is expected to skyrocket from $5.91 billion in 2018 to $198.17 billion in 2025. Market-intelligence firm Tractica estimates that the market for AR smartglasses will grow from 150,000 units in 2016 to 22.8 million by 2022, when revenues for glasses should reach $19.7 billion.
Research firm Frost & Sullivan defines augmented reality as “the art of superimposing computer-generated content over a live view of the world.” In a report on augmented reality for industrial applications, Frost says that “AR integrates digital information with the user’s environment in real time and is increasingly becoming accessible and affordable in varied applications.”
AR helps solve a crucial problem with Internet-delivered information: the mental challenge of translating two-dimensional information from a flat screen onto three-dimensional reality.
“That isn’t always easy, as anyone who has used a manual to fix an office copier knows,” wrote Michael Porter and James Heppelman for Harvard Business Review.
“By superimposing digital information directly on real objects or environments, AR allows people to process the physical and digital simultaneously, eliminating the need to mentally bridge the two,” wrote Porter and Heppelman. “That improves our ability to rapidly and accurately absorb information, make decisions, and execute required tasks quickly and efficiently.”
To understand better the potential for augmented reality in automotive service, Dealer News Today spoke with John Parkinson of tech strategy firm ParkWood Advisors. Parkinson has worked on development of AR applications. He thinks AR technology has good potential for use cases like auto service, but overlaying 2D information on 3D reality does present hardware-related challenges.
“To make AR work,” he said, “you need to use a display device—a little thing that’s close up—to look at a bigger thing that’s farther away. If you want people to see accurate imagery, contrast is a big factor. So, for example, you might need to redesign the service-bay lighting to give adequate light for smartglasses.”
Parkinson also points out that battery life is a factor to consider, as well as Wi-Fi network access in the auto shop.
“Can you get adequate network coverage in the noisy electronic environment of a car-dealer service bay? These things aren’t a slam-dunk.” All the same, said Parkinson, “The technology has improved since I started playing around with it almost 10 years ago. If you look at the trajectory of this, you’re going to eventually have something that works. There could be circumstances where it’s a good idea to try it out.”
How Bosch, Porsche, and BMW Are Augmenting Reality in the Service Department
The potential benefits of AR are confirmed by research undertaken by auto and industrial parts manufacturer Bosch. Bosch has developed its own AR platform, called Common Augmented Reality Platform (CAP). Bosch researchers found that: “Using augmented reality applications, time savings of 15 percent on average per-step-taken are possible even on common vehicles and less sophisticated repair tasks such as removing specific components.” AR allows repair technicians to “see the location of hidden components, [with] instructions or required special tools integrated into the real image,” which “clarifies the next steps to be taken, thus avoiding unnecessary disassembly and assembly work.”
CAP was previously used with tablet computers and smartphones, but Bosch recently announced that its platform now works with Microsoft HoloLens 2 smartglasses, allowing hands-free operation.
Porsche explains that, before the rollout of its Tech Live Look solution, “a complex or unusual technical issue could go back and forth repeatedly between a dealership and the PCNA technical support team. It could take multiple electronic messages, phone calls, photos, and even on-site visits by Porsche’s Field Technical Managers to identify and diagnose the issue for repair.”
The company describes a new scenario now made possible with Tech Live Look.
“A service technician at a dealership, for example in Los Angeles, dons the smartglasses and connects through the software with the Atlanta-based Porsche technical support team 2,200 miles away. Via high-definition live video from the glasses, the support team sees exactly what the technician is seeing. The expert in turn can project step-by-step technical bulletins and schematic drawings onto the display inside the technician’s glasses, as well as take screen shots and enlarge images for better visibility. The technician can open and view documents while working hands-free on the car. This information exchange is far more efficient than sending electronic forms and photos or explaining complex technical issues over the phone.”
BMW’s AR-oriented solution is part of a broader set of new technical and communication technologies the company is implementing. TIS 2.0 (Technical Information System) is a multimedia platform for technician support, designed to deliver technical information via mobile devices. Technical Support and Research Assistant (TSARA) is an online case-reporting system. TSARAVision Smart Glasses are used to connect technicians remotely to engineers and other experts at a central location in conjunction with the HMT-1 by Realwear, a ruggedized head-mounted Android device. The HMT-1s are running Frontline software by Ubimax, a solution that provides connectivity and content access between the glasses and back-end systems.
TSARAVision saves time and expense for BMW’s team of field representatives, said Frank Lampe, marketing vice president at Ubimax, speaking to Dealer News Today.
“Traditionally, BMW has had a team going around in person inspecting the more difficult cases,” he explained. “It consumes a lot of time traveling.”
With TSARAVision, BMW’s field reps can tie in remotely on a repair problem, participate in a solution, and even sign off on a warranty repair without having to travel to the dealership.
Lampe stressed that the broader TSARA system provides a large library of “support resources like manuals and instructional guides and so on, but all of these resources are not necessarily delivered over the smartglass system,” which is brought into play when needed.
As it happens, Parkinson of ParkWood Advisors drives a BMW and has had the opportunity to observe auto technicians using the TSARAVision system. He told Dealer News Today that, since he’s worked with augmented reality applications, he was interested in hearing auto mechanics’ opinions about the technology.
“Service managers seem to be of two minds about it,” Parkinson said. “The less experienced technicians love it, they tell me. They spend less time having to go to the computer to look things up. On the other hand, they tell me the guys who have been doing this work a long time already know what to do. You plug the car into the computer, and the computer tells you what to pay attention to. These guys don’t really need to refer to the service documentation very much.”
Parkinson’s findings are somewhat at odds with an industry source we spoke to who has intimate knowledge of smartglasses implementations in auto service. He told us that some technicians are reluctant to use the new technology at first, but that they become enthusiastic after trying it. The system is easy to use and saves time in the long run.
AI Makes Friends With AR
As things stand now, the BMW and Porsche implementations seem to do their best work as a way for on-site technicians to communicate hands-free with remote tech-support experts and to receive information as needed via smartglasses. However, augmented reality is quickly moving in the direction of what Michael Porter and James Heppelman call “registered” experiences that “anchor information to specific objects.” They write that “They can do this through markers, such as bar codes, logos, or labels, which are placed on the objects and scanned by the user with an AR device. A more powerful approach, however, uses technology that recognizes objects by comparing their shape to a catalog of 3-D models.”
Artificial intelligence (AI) applications can be taught to recognize physical objects, which means that the back-end software delivered to a set of smartglasses could recognize what car it’s looking at and what engine or system the tech is working on, and then automatically deliver the repair information the tech needs to do the job, without assistance from a remote engineer.
While BMW’s system is currently limited to remote support and access to documentation, Parkinson believes that the capabilities described by Porter and Heppelman aren’t necessarily that far off. Artificial intelligence models are rapidly getting better at image recognition.
“If you think about it,” Parkinson told Dealer News Today, “there’s a camera in the glasses, so the system can see what you’re looking at and could be taught to do some contextual intelligence about what it should be showing you, depending on your task. It doesn’t have to solve for every possibility. It knows you’re looking at a car, and there are only so many things on the car. If you’re working on the disk brakes changing the brake pads, the software will be smart enough to recognize the objects in your field of view and to select the parts of the manual that are relevant, based on what you’re looking at. It’s not going to show you data from some other part of the car.”
