BMW just put AR in a car, and it finally makes perfect sense

If you asked me to wear AR glasses on top of my regular glasses for a short ride in a lovely BMW EV, I would’ve said, “No thanks.” But now I’m glad I agreed because I can almost believe that augmented reality makes sense, not just for passengers but as a tool for drivers, as well.

Here in a parking lot at CES 2024 where car tech is everywhere, I hopped into an all-electric BMW iX M60 and donned a pair of Xreal Air 2 augmented reality glasses. I had to wear them over my regular eyeglasses because there was no time to put the right prescription lenses in them. Fortunately, the AR effect still looked quite good.

As soon as I put the frames on, I could see a small circular platform sitting on the dash. For this experience, BMW put me in the passenger seat while a BMW rep drove (without AR glasses) and explained the technology to me. For this demo, we drove out of the lot, away from the convention center, and onto the streets of Las Vegas.

What I saw through the glasses surprised and often delighted me. It’s the subtlety that makes it work. There’s nothing that jumps up to occlude your vision. Instead, the mostly translucent objects appear on the dashboard or the road ahead. To be clear, though, AR elements can appear anywhere you look: out the windshield, out the passenger window. I forgot sometimes that everyone else couldn’t see what the Xreal Air 2 AR glasses were showing me.

Throughout our trip, I saw green arrows directing me to my destination; they would curve left or right to alert me of an upcoming turn (comically, the driver who was not wearing the glasses managed to miss one of our turns). The idea here is that this guidance could replace your standard GPS navigation system. Granted, the system is connected to a phone that is connected to the car. Still, I didn’t have to glance down at another screen to see where I was going.

In addition to the direction queues, there were these-colored blocks that would appear on either side of the car. They were green for all clear, yellow for caution, orange for danger, and red for stop. If there was a stop sign ahead, an AR one would appear before me. There were virtual caution cones to warn me of potholes and other road hazards.

The system also accommodates games and entertainment for your passengers. At one point, the sky started raining virtual dice (BMW preprogrammed and timed these engagements to specific moments in our trip). Later there was a game where virtual poker chips were flying toward me. I earned points for each one I managed to, with my gaze, guide through a floating ring. I only managed 6 points.

On the virtual control panel, there was a media button. When we triggered it, the BMW rep told me to flip down my visor. On the surface was a virtual screen that looked like an 80-inch display sitting about 3 feet away from me, and it was playing, naturally, a BMW commercial. I was surprised and appreciated the image quality. 

At the end of our trip, we pulled back into the lot, and in front of me, I could see guidance to a charging station. The AR graphic showed how much charge I had left and how long it would take to recharge the iX M60.

One thing I noticed about all the imagery is that it understood the surfaces and chassis of the EV. BMW told me that it was not using the Xreal’s camera to read the surfaces. Instead, they had added a cabin camera to watch where I was looking properly positioning the control panel on the dash, and managing occlusion.

The latter is what really sells the AR effect. Those guidance blocks I mentioned disappear behind the edge of the car hood. In other words, they appear as real objects and not in some illusion-crushing and distracting form where I could oddly see the entire blocks through the body of the car.

This is the key to successful AR in a car. It not only has to provide an augmented experience, but it has to work with and make sense in the real world.

BMW has no current plans to add AR glasses or transparent AR windshields to all its EVs. For now, it’s a research project and what they learn here may inform the future of your driving experience.

We’re covering all of the latest CES news from the show as it happens. Stick with us for the big stories on everything from 8K TVs and foldable displays to new phones, laptops, smart home gadgets, and the latest in AI.

And don’t forget to follow us on TikTok for the latest from the CES show floor!