‘In 2017, we didn’t think it was possible to stream VR. But two years later, we launched it.’ Gregory Jones, head of XR products team at NVIDIA, speaks to Health Tech World about its role at the very forefront of the fast-developing world of VR and its world-leading innovation to support students
Through NVIDIA’s sector-leading work in accelerated graphics since its formation in 1993, confident that one day the PC would become a consumer device for enjoying games and multimedia, it is now at the very forefront of the next phase of its development.
With the creation of its groundbreaking CloudXR, NVIDIA is now delivering VR and AR across 5G and Wi-Fi networks – and in a huge boost to the future of healthcare, the business has created a world-first project in redefining possibilities for how students learn.
Working alongside Coventry University, Vodafone and The Grid Factory – a UK-based provider of NVIDIA GPU-accelerated services – student nurses and healthcare professionals can experience lessons and simulations done in VR environments on non-tethered headsets.
Rather than the need to date for physical connection to a high-performance computer that drives rich, immersive environments, instead it streams from NVIDIA RTX-enable servers located in the cloud or on premises, which deliver the advanced graphics performance needed for wireless virtual, augmented or mixed reality environments — which collectively are known as XR.
The creation of this project, which is now set to be replicated in other environments around the world, has been the latest step on the ongoing NVIDIA journey in pushing boundaries.
“Often discoveries like this start as a glimmer, an idea of what we could do, and then we can work on it to bring it to reality. It’s been a fun journey,” says Gregory Jones, who manages the XR products team.
“What’s interesting here is that in 2017, we didn’t think it was possible to stream VR. It’s a hard fact that when a user puts on a headset and turns their head, we have to update the graphics within 20 milliseconds, it’s really fast, and taking longer than 20 milliseconds causes nausea.
“We came up with the idea of the moving ‘late latch’ to the client device graphics processing unit (GPU) to create graphics experiences where the users gets updated information in under 20 milliseconds, it’s a really clever way of overcoming the graphics latency problem, which then makes streaming VR possible.
“So we’ve gone from ‘We can’t stream VR’ to ‘We can stream VR really effectively over 5G’.”
And the application of such a sector-changing breakthrough holds just as much potential – remote learning is now being enabled in better quality than ever before, with streaming via 5G enabling lecturers to teach in VR while students can access the interactive environment through smartphones, tablets, laptops, and eventually their own VR headsets and AR glasses.
With Coventry University’s advanced VR technology, users can now take virtual tours through the human body. Previously, lecturers would have to use pre-recorded materials, but this only allowed them to view the body in a linear, 2D format – now they can guide students on interactive explorations of the human body in 3D models.
In the immersive environment, detailed parts of the body, from the bones, muscles and the brain, to the heart, veins, vessels and blood cells – as well as detailed simulations of viral attacks – can all be brought to life.
“Explorers like those at Coventry University are so incredibly important in the evolving ecosystems. They spend money, take a risk, and help to create something really meaningful. We love risk takers,” says Gregory.
“They are showing how students can have more meaningful, impactful lectures through the interaction of 5G networks. This is the first place in the world to be doing this.
“With the delivery of medicine and treating humans, it’s very often a passion rather than just learning. What they could see previously on a small screen they can now get to see in 3D in VR, they can see the human body in full scale – and that passion can be even more expansive in experiencing the beauty and also the subtlety of the human body through this kind of interaction.”
The adoption of such modelling techniques in medicine and education build on previous techniques which formed the previous stage of VR development and its application, says Gregory.
“3D modelling has been around for years and we see VR helping in that area of design,” he says.
“VR builds that same model with graphics, to the extent you can see a full scale body with anatomy in real size.
“With architecture, if you’re buying a house, you can pick your surfaces and beautiful lighting months before it is built, you can see a 40ft wall before it’s there. You can see these things at scale through 3D graphic modelling.
“It’s the same in building a human body, and we can now use VR in a similar way to visualise the anatomy models.”
With projects like this showing what is possible, Gregory predicts an exciting future for VR in its wider adoption in the next few years, through the roll-out of the public 5G network.
“We’re seeing a couple of phases of growth – the first is in the data centres and factories, we’ve got a whole team of folks in Dell and Ericsson working in their private data centres, so we’ll expect to see growth there in the next two to three years,” says Gregory.
“But we’re also seeing really pervasive use of AR and VR among the consumer. When VR was first created, we saw a relatively organic growth rate, sales of headsets were 30 per cent year on year, which is good growth but not explosive.
“At first, you’d need a tethered set with a very expensive GPU in the workstation, but then with the Oculus Quest 1, which was reasonably priced, they saw a 90 per cent growth rate. The Quest 2 is said to be even faster. We’ll see more devices in the next five to six years with the public 5G network growth.
“Now we can stream from a remote server with an NVIDIA graphics card and you can get the really high end graphics with the ease of use of headsets like the Oculus Quest. Seeing that combination is really extending that growth into explosive growth – this is really the tip of the iceberg.”