We are witnessing a digital age where virtual and physical worlds are converging and renovating the landscape for many industries. Extended Reality is offering us new ways to interact with products and services, boosting customer engagement and loyalty.
Technology, as we all know, has the power to alter the means of how we operate in the physical world. ‘The Matrix’, 1990s classic movie depicting people living in a virtual world seemed so unrealistic back then. What was once considered as science fiction in books and movies is becoming a reality today. Extended Reality solutions being developed today have the potential to create the same immersive experience as we’ve seen in our favorite sci-fi titles.
What is Extended Reality (XR)?
Extended Reality is an umbrella term incorporating Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR). It is referred to as XR where “X” could be considered as a variable that may be replaced with any of the letters “V”, “A” or “M”. Accordingly, Extended Reality could be defined as the use of immersive technologies that extend reality and merge both physical and virtual worlds.
Virtual Reality interchanges the real-world environment completely and constructs a whole new digital world of the user. It could be experienced through VR-headsets, or head-mounted display that allows a 360° view of the digital world.
Examples include virtual experiences such as running on the moon, jumping from one building onto another or walking over the clouds. For instance, for its luxury SUV XC90 model, Volvo introduced the VolvoReality that allows consumers to experience a full VR test drive using their phones and a Google Cardboard headset.
Augmented Reality bridges the real world environment of the user with the constructed digital environment, thereby enhancing the real-world with digital details such as images, text and animation. It could be experienced through AR-glasses, tablets and smartphones.
Examples include ‘Pokémon Go’ game that overlays digital creatures into the real world, and Snapchat filters which add digital layovers to your photos such as glasses or a hat. L’Oréal, the famous cosmetics company, launched a makeup virtual try-on feature that allows users to test different types of makeup using AR.
Mixed Reality combines both the digital and the real world into one environment so they coincide and interact with each other in real-time. This could be experienced through MR-headsets.
One of the examples is Microsoft HoloLens, a MR headset, that allows to place a digital object and to interact with it in a defined way. Case Western Reserve University piloted the initial HoloLens testing by enabling its students to examine human anatomy. Students were able to see, walk around, and dissect the digitally recreated organs which boosted their learning experience.
Top Industries Transformed by XR
The following sectors have adapted XR technology with astounding results:
The retail sector is getting smarter by enabling the customer to experience the use of a product before it’s actual acquisition. Rolex, one of the leading watchmakers worldwide, has adopted AR to provide its customers the luxury of trying out virtual watches on their wrist.
This creates innovative prospects to connect and get potential clients to try out their products remotely, and expand business. Also, there is a marked reduction in returns from e-commerce customers.
AR can also be adopted to provide additional information about the product as it was done in the wine industry.
Another good example is IKEA, the Swedish furniture company, which enables customers to place virtual furniture items into their home via their smartphone using AR.
Healthcare provides boundless opportunities for XR implementation. The technologies are used to distract patients from pain, create training materials, and even make diagnoses.
Karuna Labs is a virtual training program with a unique approach to chronic pain treatment and rehabilitation. OxfordVR provides psychological treatment using a truly immersive experience.
Education & Training benefits most from using XR solutions. NASA is actively experimenting with XR to train its workers in space station repair and maintenance. At Boeing airplanes, electrical technicians make use of AR-headsets to check maintenance checklists which reduced their inspecting and work time by 25%.
Agco, a leading agricultural equipment manufacturer, implemented an AR-training program to support its quality assurance initiatives. The US Marines have used a system called “Augmented Immersive Team Trainer” that allows soldiers to be trained virtually by adding different fire-effects and enemies that has significantly enhanced their performance in the real-world scenario.
Marketing is another major industry affected by XR technology. Advanced immersive marketing techniques allow the customer to try out products and services in a virtual environment. For instance, “Holoroom How To” places marketers in a fully-immersive virtual reality skills training ground.
AMC Cinemas, a US movie theater, released an AR-app that allows users to view a movie trailer just by pointing their phone at a poster. Starbucks Cup Magic app is another promising AR-app which allows the user to send and receive virtual Valentine messages and Starbucks gift cards.
Real estate XR solutions allow agents to send home buyers on virtual tours providing the luxury of exploring the houses without having to physically visit the site. Potential buyers or renters can virtually visit dozens of locations In under just a few hours and decide which are worth visiting in person.
Entertainment also benefits from XR technology by discovering more ways to engage with the audience. For instance, Treadwater comic books series owners introduced an AR mobile app that allows readers to interact with characters on pages of tread-water graphic novel hard copies.
How does XR work?
XR makes use of the device’s central processing unit (CPU), camera and software to alter the surrounding perception of the viewer. The software uses the CPU to process the images collected by the camera and create a digital environment for the user to interact combining elements of both the real and the virtual world.
Let’s go through the process in more detail:
- Real word data input. XR relies on Inertial Measurement Units (IMU) made up of accelerometers and gyroscope sensors and has an embedded magnetometer. The tracking information collected by these components are used to estimate the drift and movement of the user. By adding optical detection with computer vision algorithms, XR solutions enable Six degrees of freedom (6DoF) movement tracking.
- Data Processing & Image generation. Powerful CPUs are required to process information collected from IMUs at rapid speeds. In more advanced systems, multiple core graphics processing units (GPU) are used for computations & graphical operations. Spatialized audio and ambisonics sound format forms the basis audio output for the 3D environment to provide a more realistic XR experience.
- Output: As soon as barrel distortion is ready it is possible to show each of your eyes a different view. However, some of the fields of view that our eyes normally see are still missing. To restore this part, lenses can be used by introducing the “pincushion distortion” effect. Then “barrel distortion/compression” and “pincushion distortion/ decompression” cancel each other making the peripheral image more realistic.
Finally, XR technology makes use of Bluetooth to communicate sensory tracking information with the processing system.
XR Technology Challenges
Some of the challenges facing XR technology are summarized below:
- Screen Door Effect. This is the visual effect that allows fine spaces between pixels to be seen due to a large image scale. However, this can be resolved through advances in both lenses and screen. Nguyen et al., have explored the effect of “shaking” the display with a square pixel architecture via pixel actuators. This method shifts the pixels which reduces the gap between them by enlarging their apparent size. Alternatively, air-gap-embedded flexible hazy films can reduce the screen-door effect.
- Motion to photon latency. This is the delay between the head movement of the user and seeing the change in the virtual world as a display of the XR device imitating the user’s movement. This can cause motion sickness. As a solution, researchers have developed a six degree of freedom model for user movement prediction which reduces the positional rendering error. The translational and rotational motion of a user can be tracked using one or more sensory data that are processed and fed to a warp engine that updates the rendered image before sending to a virtual reality display.
- Safety Concerns. If the app uses an XR controller then the user may collide with other objects causing harm at both ends. Moreover, the user may trip over an obstacle while moving around causing the user to lose their balance. Intel has put together guidelines on how to create a safe, comfortable, and immersive VR experience.
The Future of XR
Envisioning futuristic XR systems is now possible. XR extends beyond Visuals and Audio to cover Haptics as well. Haptics involves technologies that use our sense of touch, i.e., haptic devices are touchable virtual reality. For instance, Logitech Rumblepad is a game controller embedded with onboard haptic feedback in a game. A range of haptic devices are currently available like the haptic gloves, six degrees of freedom haptic controllers, haptic suits, locomotion controllers and haptic boots.
Magic Leap’s, Vive Pro is an innovative step in XR hardware development. It has a light field photonic feature that will allow virtual objects in XR to exist at any depth in the user’s view. The eye-tracking feature of the XR headsets allows the device to know where the user is looking so that area is presented at higher resolution compared to the surrounding area. The latter has been improved further by introducing the Fixed Foveated Rendering (FFR). XR devices will also support gesture recognition that enhances XR functionality.
Conclusion
Extended Reality (XR) tech is making breakthroughs in finding new ways to interact with consumers. XR solutions are augmenting and increasing productivity of many industries, including real estate, retail & eCommerce, education & training, entertainment, and more. In a more distant future we may really see a world where virtual interaction prevails over real-life experiences.
