In this column, we briefly introduce the concepts of virtual reality (VR), mediated reality (MR), mixed reality (MR), augmented reality (AR), diminished reality (DR), augmented virtuality (AV), and hyper reality (HR).
There are so many new technologies bouncing around these days that it’s difficult to wrap one’s brain around all the terminology and abbreviations, so I thought it might be a good idea to provide some easy-to-read overviews.
In this column, we are going to briefly introduce the concepts of alternative realities, virtual reality (VR), mediated reality (MR), mixed reality (MR), augmented reality (AR), diminished reality (DR), augmented virtuality (AV), and hyper reality (HR).
Alternative Realities
Since time immemorial (see Is Time Truly an Illusion?), humans have been entranced by the thought of realities other than our own. Imagine the scene thousands of years ago in which a group of people are huddled around an open fire at night, listening to their storytellers weave tales involving gods and supernatural entities, daring deeds and fabulous failures, heroes and heroines, winners and losers, and monsters and mythical creatures.
Every time we open a book, watch a movie, or see a play, we are transported to an alternate reality. Occasionally, some amongst our number decide to help things along, as did the anthropologist Carlos Castaneda with psychotropic plants like peyote and interesting mushrooms of the genus Psilocybe. Amazingly enough, Carlos somehow managed to document his experiences in his book A Separate Reality.
Excluding things like movies, one of the earliest examples of using technology to create an alternative reality was called the Sensorama. This little beauty was created in 1962 by Morton Heilig, who was a producer, director, writer, cinematographer, and editor of films and TV programs.
The Sensorama offered an immersive, multi-sensory (now known as multimodal) sensory experience. Primarily a mechanical device, the Sensorama boasted a stereoscopic color display, fans, odor emitters, a stereo‐sound system, and a motional chair. It simulated a motorcycle ride through New York and created the experience by having the spectator sit on an imaginary motorcycle while experiencing the street through the screen, fan-generated wind, and the simulated noises and smells of the city. All of these elements were triggered at the appropriate times, such as the smell of the exhaust when the rider approached a bus. The petrol fumes and the aroma of pizza snack bars were recreated using chemicals.
I don’t know about you, but I would love to take a ride on the Sensorama. If I can ever get my time machine working (again), I would also love to bring Morton Heilig forward to the present day and let him play with my Oculus Rift virtual reality system, but we digress…
The Future is Closer Than We Think
Arthur C Clarke was a prolific writer and a brilliant futurist, and he is widely known for the third of his famous three laws, which states: “Any sufficiently advanced technology is indistinguishable from magic.” He was right; these days we have access to technologies that would have seemed the stuff of magic not so long ago. Let’s take a brief look at a few of these little rascals…
Physical Reality (PR): This refers to the real world we see around us, although the more I read, the harder it becomes to determine what is real and what isn’t.
In his book, Reality is Not What It Seems: The Journey to Quantum Gravity, Carlo Rovelli works his way from Sir Isaac Newton with his space, time, and particles to Faraday and Maxwell and their fields to Einstein and his spacetime, ending up with the current theory of Covariant Quantum Fields.
In a nutshell, it now appears that we are all swimming in a sea of quantum foam — there really isn’t such a thing as space that “contains” things and there isn’t really such a thing as time during the course of which events occur. I know this sounds silly when you say it out loud, but it makes a lot more sense when you read the book.
Virtual Reality (VR): In this case, the reality is completely generated by a computer. VR has tremendous application in industry for training, such as teaching crane operators how to manipulate heavy loads in adverse conditions. As one example, consider the task of loading cargo onto a boat in high winds with a heavy swell in limited visibility. Preferably, you would wait for conditions to improve, but you might not have this luxury in the case of a mission-critical situation like an emergency aid deployment. It’s much better to practice this stuff in the virtual world than to take chances with real boats, cranes, and people.
VR is also of interest for education and entertainment. I personally enjoy a quiet afternoon cracking puzzles (see Solving Multifaceted Mysteries in VR), surviving a zombie apocalypse (e.g. Arizona Sunshine) or exploring alien worlds (e.g., Obduction).
Of course, this is going to be so much better when we are no longer tethered to a honking big host computer (see Time for an Oculus Quest?). Also, we are going to see increasing use of VR in conjunction with artificial intelligence (AI) (see AI, ANNs, ML, DL, and DNNs).
Games like Star Trek: Bridge Crew already make use of AI technology. In this case, the idea is that you are part of the bridge crew flying a starship on various missions. This crew comprises four positions: Captain, Helm, Tactical and Engineer. You can either play with your friends or — on the off-chance you don’t have any — you can populate the other positions with AI-enabled characters.
Augmented Reality (AR): This refers to an interactive experience of a real-world environment in which the objects that reside in the real world are enhanced by computer-generated perceptual information, sometimes across multiple sensory modalities, including visual, auditory, haptic, somatosensory, and olfactory.
Diminished Reality (DR): Also known as deleted or deletive reality, DR is the complement of AR. As opposed to adding information or stimuli to a real-world scene, DR involves removing or diminishing information or stimuli from the real world. Examples would be to fade down (or out) extraneous voices or other sounds when you are conversing with someone in a noisy environment, fading or blurring portions of the scene you are viewing, or completely removing objects or people from the reality with which you are engaging.
Mediated Reality (MR): This refers to adding information to (augmenting), subtracting information from (diminishing or deleting), or otherwise manipulating one’s perception of reality. This can be achieved by means of a wearable computer (e.g., a headset) or a hand-held device (e.g., a smartphone).
I personally believe that, in the not-so-distant future, the combination of MR (i.e., PR + AR + DR) with AI is going to dramatically change the way in which we interact with the world, our electronic systems, and each other.
Augmented Virtuality (AV): As opposed to AR, in which objects and scenes in the real world are augmented with computer-generated information, augmented virtuality refers to augmenting virtual environments with real-world objects or people.
Mixed Reality (MR): This refers to merging of real and virtual worlds to produce new environments and visualizations, where physical and digital objects co-exist and interact in real time. Mixed reality does not exclusively take place in either the physical or virtual world, but is a hybrid of reality and virtual reality.
Reality-Virtuality Continuum: As illustrated in the diagram to the right, this refers to a continuous scale between 100% physical reality (PR), the real world, at one end of the spectrum, and 100% virtual reality (VR) at the other end.
Is mediated reality (MR) really a superset of mixed reality (MR)? To be honest, I would personally say they are one and the same thing, but those people who are fond of presenting things in the form of Venn diagrams usually depict mediated reality as encompassing mixed reality. It’s also probably worth noting that different people may have slightly different interpretations of a lot of this, plus folks are constantly adding new terms that muddy the waters (not that I’m bitter, you understand).
Hyper Reality (HR): First, we have to distinguish hyperreality (one word) from hyper reality (two words). The former, hyperreality, is an inability of consciousness to distinguish reality from a simulation of reality, especially in technologically advanced postmodern societies. Hyperreality is seen as a condition in which what is real and what is fiction are seamlessly blended together so that there is no clear distinction between where one ends and the other begins. Hyperreality also allows the co-mingling of physical reality (PR) with virtual reality (VR) and human intelligence (HI) with artificial intelligence (AI).
By comparison, hyper reality (HR) refers to a total information overload, which we might think of a mediated reality (MR) on steroids. Perhaps the best way to visualize this is to look at the Hyper Reality concept film by Keiichi Matsuda (it’s only ~6 minutes long).
This amazing video depicts a kaleidoscopic vision of the future in which physical and virtual realities have become totally entwined. On the one hand, it looks exhilarating; on the other hand, I don’t think my poor old noggin would be able to take the strain. As always, what’s your take on all of this?
