Seeing each other in times of Covid
Markus Gross is convinced that there are better technologies available than today's video conference systems. The key technology is developing at a rapid pace – but not fast enough.
Imagine you simply say to the system, “Call my parents” ?– who are perhaps physically far away or not allowed to visit because they belong to a risk group – and your parents’ photorealistic avatars appear and sit down on the sofa next to you to have a conversation. All science fiction? No, we are not too far from such a reality because the technology is almost there.
These surreal times of social distancing, personal isolation and excessive use of video conferencing have once again highlighted the significant limitations of today’s video conference systems. No matter which one I use – Zoom, Skype or Hangouts – after some time into the conversation, I become stressed, bored or tired.
A holodeck – similar to Star Trek
It reminds me of an old paradigm dreamed up in research labs over the past 50 years and throughout the last 100 years of cinema: the idea of a comprehensive, holographic telepresence. In fact, it's a kind of magical, futuristic holodeck system that teleports people so they can meet and chat over long distances, potentially in virtual worlds. Such a technology would give users a strong sense of presence, of sharing the moment with the person with whom they want to interact.
As a researcher in the field of visual data processing, and before my time at Disney, I worked on several concepts of immersive telepresence. Inspired by the vision of my dear friend Henry Fuchs and his external page OfficeOfTheFuture, I embarked on a challenging journey in 2000 to build Blue-C. Back then, Blue-C was the world’s first holodeck with stereoscopic 3D projection and holographic video. It was a heroic engineering feat in a time of limited graphical, computing and network resources, and we worked for three years to complete the prototype. The project also brought my doctoral students almost to despair at times: “Markus, this darned thing will never work!” Well, after numerous attempts it did somehow work and gave us very valuable insights into what makes a telepresence system really believable.
Being there and future systems
Ten years richer in experience and age, together with partners from the University of North Carolina and the Nanyang Institute of Technology, we launched the next generation research platform for tele-immersion: BeingThere. The project drove forward all the key technologies required to construct teleportation systems: holographic video, 3D display, eye correction, digital avatars and much more.
Industry has also made progress in this area and a large number of commercial products are already on the market. Companies such as Cisco have a long history of development of telepresence systems; however, these are more an improved version of existing telepresence tools than a real technological breakthrough that would change our lives for the better.
"I can only encourage all would-be entrepreneurs to seize this opportunity and do something to improve this technology – we have never needed it more urgently than now."Markus Gross
I am hopeful that a solid and credible telepresence will be a reality sooner rather than later. We’re currently seeing the fundamental key technologies develop very quickly – I'm thinking of artificial intelligence, real-time computer image and machine learning, realistic 3D graphics, high-resolution 3D capture, digital people and avatars, light field displays and mixed reality platforms. For a foretaste of what will come, look at Microsoft’s Room2Room project and its holoportation platform, or the digital avatars from Facebook Reality Labs.
Pandemic makes urgency visible
The time for telepresence@work is here and now. The pandemic has made teleconferencing tools an irrevocable part of our professional and personal lives, and we need fast and usable solutions. This is the right time for all start-ups and investment funds to turn to XR technology products for immersive telepresence. I can only encourage all would-be entrepreneurs to seize this opportunity and do something to improve this technology – we have never needed it more urgently than now.