Audi Brings Augmented Reality to the Head-Up Display

Augmented reality is a long-promised capability that is finally coming to life in the Audi Q4 e-tron EV. Head-Up Displays have provided basic information such as vehicle speed that is projected onto the inside of the car’s windshield so that it is directly in the driver’s view while driving.

The possibilities for augmented reality in this application are immense, as such a system would be able to overlay the driver’s view of the real world through the windshield with helpful navigation directions, warning signs, and vehicle data. Now, Audi is delivering on that potential in the Q4 e-tron.

The Audi system displays turning arrows from the navigation system and starting points and destinations that are visually superimposed in the corresponding place on the real-life outside world as dynamically displayed augmented reality content.

Images appear to float in space about ten meters ahead of the driver. This keeps the focus for the information close to that for the real-world scenes. Audi says that the overlaid information conveys information very quickly without being confusing or distracting.

The augmented reality image appears the size of a 70-inch big-screen television at the focal distance.

Below the augmented reality imagery is what Audi calls the “status” section of the display. That’s where traditional HUD-provided data about the car is shown, so drivers see information there like vehicle speed and traffic sign information.

Related:Audi's 2020 Technology Hits

There is a good reason why augmented reality HUDs have taken a while to mature: it is hard to do! Audi’s system relies on what the company calls a “picture generation unit.” That device has to fit inside the vehicle’s dashboard among the HVAC vents, vehicle controls, instrument panel, and infotainment displays.

It uses a bright LCD shining onto two parallel mirrors, with special optical components to separate the portions for the near-field and distant areas.

The mirrors direct the light beams onto a large adjustable concave mirror that directs the images onto the windshield. That’s where the driver sees the augmented reality, reflected in the glass.

The images come from the vehicle’s Augmented Reality Creator. This is a multi-module processing unit that also powers the infotainment system. It renders the display symbols at 60 frames per second and adapts them to the geometry of the projection optics. At the same time, it calculates their location in relation to the environment, on which it obtains information via the raw data of the front camera, the radar sensor, and the GPS navigation.

There are about 600,000 lines of programming code in the system. Audi likes to point out that is about half again as much as was used by the early Space Shuttle for flight control.

The AR Creator has to take into account fractions-of-a-second delay between the identification of an object by the sensors and the display of the information on the HUD. During this brief gap, the Q4 e-tron is moving, so the AR Creator predicts that movement to ensure that the display overlays reality accurately.

It bases that prediction on data from the camera, the radar, and the sensors of the stabilization control. This lets the system apply “shake compensation,” a few milliseconds before the picture is projected, preventing disruptive shaking of the display. Such refinements suggest that this technology will not disappoint drivers when they try it for the first time thanks to the elimination of lag-related errors that could diminish confidence in the system.