Mixed reality beckons, but are we ready?

Clive 'Max' Maxfield

October 27, 2022

7 Min Read
AR-2HCBTWK-web.jpg
Using augmented reality eyeglasses in industry.Image courtesy of Westend61 GmbH / Alamy Stock Photo

For some time now, I’ve been pondering the concept of reality, and I thought I’d share my meandering musings here.

Way back in the mists of time, our ancestors gathered around the campfire to share tales of heroes and gods and myths, oftentimes trying to explain the universe and our position in it. These days, of course, as opposed to sitting around a fire, we are more likely to be found reading a book or watching a movie or attending a stage play. I don’t think it’s taking things too far to suggest that, however it is we are drawn into the tale, we are—in some small way—experiencing a different reality.

Of course, some people have tried to help things along, such as the American psychologist and author Timothy Francis Leary, who wasn’t shy about advocating the use of psychedelic drugs. There's also the American anthropologist Carlos Castaneda, who did more than dabble in psychotropic plants like peyote and interesting mushrooms of the genus Psilocybe.

Now I’m thinking of the 1895 French short documentary film directed and produced by Auguste and Louis Lumière entitled L'arrivée d'un train en gare de La Ciotat (which translates as The Arrival of a Train at La Ciotat Station). This 50-second silent film shows the entry of a train pulled by a steam locomotive into the train station of the French southern coastal town of La Ciotat, near Marseille. The story goes that when the film was first shown, the audience was so overwhelmed by the moving image of a life-sized train coming directly at them that people screamed and ran to the back of the room. Some historians think this is a rumor, but I can easily believe it.

If you search on YouTube, there are a number of videos of seniors trying VR for the first time. The first few seconds of this offering show one of the ladies undergoing a game called Richie’s Plank Experience. The idea behind this is that, while wearing your virtual reality headset, you enter a virtual elevator and ascend about 30 floors. When the doors open, you are presented with a plank that extends out over the road 30 stories below. I’m ashamed to say that, even though I know this isn’t real, I still find it hard to walk out on that plank myself.

One Christmas when I was a kid, I was gifted a View-Master from my parents. I bet you had one too. Invented in 1939, this little beauty came equipped with thin cardboard disks carrying pairs of stereoscopic 3D photographs on film. When you brought the red plastic contraption to your eyes, you were presented with a series of glorious colored images of famous monuments or landscapes or tourist attractions.

One of the earliest known examples of immersive, multi-sensory (now known as multimodal) technology was the Sensorama, which was invented by Morton Heilig in 1962.

sensorama-HX977G.jpg

As we read in the Wikipedia: “The Sensorama was a mechanical device, which includes a stereoscopic color display, fans, odor emitters, stereo‐sound system, and a motional chair. It simulated a motorcycle ride through New York and created the experience by having the spectator sit in an imaginary motorcycle while experiencing the street through the screen, fan-generated wind, and the simulated noise and smell of the city. These elements are triggered at the appropriate time such as the case of the release of the exhaust chemicals when rider approached a bus. The petrol fumes and the smell of pizza snack bars were recreated by chemicals. While the machine still functions today, audiences cannot interact with it, and it cannot respond based on the user's actions.” Although the Sensorama may appear somewhat “clunky” to us today, I bet it was mind-blowing back in 1962.

I was first in line when the Oculus Rift virtual reality (VR) headset first hit the streets in 2016. I’d experienced VR before in the form of industrial and military systems, but I never expected it to become available commercially—at least, not at the Oculus Rift level of sophistication.

These days, there’s also a lot of talk about augmented reality (AR), but VR and AR are only part of the picture (no pun intended). The way I think about things can be summarized as follows. At one end of the reality continuum, we have the real world, which we might call physical reality (PR). At the other end of the spectrum, we have VR, in which everything we see and hear is generated by computer.

max-0034-fig-02-different-realities.jpg

AR involves taking what we sense of the real world and augmenting this with additional information. This additional information may be presented visually as text or graphics, but it can also be presented in the form of sound, smell, or some form of haptic (tactile) mechanism.

What many people don’t realize is that AR is only one side of the coin. The other is diminished reality (DR) whereby elements are diminished or deleted from the real-world scene. This may occur in the form of fading down some of the sounds, blurring some of the image or changing its color palette (perhaps making most of the image grayscale and leaving only areas of interest in color), or even completely removing objects or people from the scene.

There’s also something called augmented virtuality (AV), in which real-world elements like objects or people are added back into an otherwise VR scene. Anything that’s not pure PR or VR may be classed as mixed reality (MR).

There’s some tremendously exciting work going on in this area. For example, as I wrote in Kura’s AR Glasses are Meta-Droolworthy, a company called Kura is working on a set of AR glasses that are presented in a regular eyeglass form-factor while offering a mindboggling 8K x 6K per eye resolution (that’s about 50M pixels per eye) with an unlimited depth of field! Regarding an alternative to eyeglasses, as I wrote in Did the Metaverse’s Holy Grail Just Arrive?, there’s a company called Swave that describes what they are working on as follows: “Our HXR technology is the Holy Grail of the metaverse, delivering lifelike, high-resolution 3D images that are viewable with the naked eye, with no compromises. HXR technology enables 1000x better pixel resolution with billions of tiny, densely packed pixels to enable true realistic 20/20 vision without requiring viewers to wear smart AR/VR headsets or prescription glasses.”

Have you seen the AR app created by Google that you can run on your smartphone? When you are in a foreign country and you see a sign, for example, you can look at it through your smartphone’s camera on its screen and, although the boundaries of the sign remain as is, the words change to be in your own language. I can easily visualize a time in the not-so distant future when, if someone is travelling abroad, their MR headsets automatically translate any written and spoken words into the language of their choice.

I honestly believe that MR is coming, and that it will be with us sooner than we think. When I look back on the ways in which technology has changed in the past 50 years, and when I consider how things seem to be accelerating, I think MR at the level we currently regard as being the stuff of science fiction will be with us sometime in the next 10 to 20 years. And, when this does come into fruition, I believe it will dramatically change the ways in which we interact with our systems, the world, and each other. How about you? Do you have any thoughts you’d care to share on this?

About the Author(s)

Clive 'Max' Maxfield

Clive "Max" Maxfield is a freelance technical consultant and writer. Max received his BSc in Control Engineering in 1980 from Sheffield Hallam University, England and began his career as a designer of central processing units (CPUs) for mainframe computers. Over the years, Max has designed everything from silicon chips to circuit boards and from brainwave amplifiers to Steampunk Prognostication Engines (don't ask). He has also been at the forefront of Electronic Design Automation (EDA) for more than 35 years.

Well-known throughout the embedded, electronics, semiconductor, and EDA industries, Max has presented papers at numerous technical conferences around the world, including North and South America, Europe, India, China, Korea, and Taiwan. He has given keynote presentations at the PCB West conference in the USA and the FPGA Forum in Norway. He's also been invited to give guest lectures at several universities in the US and at Oslo University in Norway. In 2001, Max "shared the stage" at a conference in Hawaii with former Speaker of the House, "Newt" Gingrich.

Max is the author and/or co-author of a number of books, including Designus Maximus Unleashed (banned in Alabama), Bebop to the Boolean Boogie (An Unconventional Guide to Electronics), EDA: Where Electronics Begins, FPGAs: Instant Access, and How Computers Do Math.

Sign up for the Design News Daily newsletter.

You May Also Like