Microsoft Mesh: Mixing the Digital World with the Real World

Mesh is a new collaborative mixed reality platform from Microsoft. (Image courtesy of Microsoft.)

Mesh is a multi-device communication platform that allows users from different locations across the world to collaborate in a shared 3D space. Mesh operates by pairing desktop or mobile devices with virtual reality (VR) headsets and holographic projections to create a unified experience. Whether it is designing, training, conducting project meetings, or attending a simple social gathering, users will be able to project avatars or photorealistic holograms of themselves that can share and interact with 3D/2D content, as well as communicate with their teammates as if they are in the same geographic location.

Mesh’s launch was rife with this blend of reality. Alex Kipman, technical fellow at Microsoft, led the keynote presentation—not standing on a stage as you would expect— but in an immersive, underwater 3D VR setting. The audience joined the event from all over the world as 3D avatars of themselves whose likes and other emojis bubbled into the presentation in real time. Guy Laliberté, the cofounder of Cirque du Soleil, joined the presentation as a guest speaker—not in person but as a holographic avatar of himself. He concluded the presentation with a Cirque du Soleil performance rendered entirely in a VR island setting with holographic performers and 3D avatars of the audience.

Not-so-Magical Realism

Combining the physical world with the digital world is known as mixed reality (MR). As Industry 4.0 and computer vision continue to advance rapidly, our ability to interact with the digital world through MR devices is not only possible but already happening.

Video games like Pokémon Go are an example of MR where digital characters exist in the players’ physical geographic locations. Online concerts and fitness apps frequently employ user avatars to allow interaction with other users online. Gucci has developed an app that empowers its customers to scan their feet and select the shoes they are interested in—all without needing to visit the store in person. In 2020, Mercedes-Benz introduced Virtual Remote Support, which allows technicians to connect with expert service representatives and address any issues they are experiencing with their cars. The technicians can show the service rep exactly what they are seeing in real time using virtual reality goggles, enabling the technician to scale the image, mark it or even use lines and other shapes to highlight specific areas. Using Virtual Remote Support has allowed Mercedes-Benz to reduce costs and time delays while eliminating errors that would otherwise have occurred. MR is also being implemented in the oil and gas sector, where it is used in design stages as well as for training end users.

With Mesh, the scope of the MR is both expansive and interactive. Mesh is powered by Azure, a public cloud platform that grants users the chance to create, design, test and manage applications. It offers more than 200 cloud products and services not limited to AI, machine learning, mixed reality and Internet of Things (IoT), all of which can be integrated into the platform.

Please click HERE to view the original article.