The art of the possible - XR training

As part of our ongoing series talking to some of the most fascinating minds in defence technology, this week we spoke with Dr Helen Dudfield, chief scientist for training and human performance at QinetiQ.

Helen has worked with QinetiQ for over 20 years now and has spent the best part of the last decade developing strategy and researching future learning environments, as well as how technology can solve the human science challenges that present themselves in complex work environments.

She started as a research psychologist in the 90s before joining the Royal Aircraft Establishment where she helped design interfaces for pilots using academic research and controlled trials. Even back in the halcyon days of the late 90s, they were already experimenting with virtual reality (though Helen admits it was "very heavy and very uncomfortable").​

Today, the technology available means she can use virtual reality (VR) and augmented reality (AR) to train individuals and teams in a manner that wasn't practical around the turn of the century. As Helen says herself: "We've gone from an interface between the human and the technology to an interface between humans and the future."

Blending, not replacing - Human/machine futures

"What we're working on currently," Helen begins, "isn't a passive interface between a human and a machine but a situation where the machine becomes more like a teammate; moving from interface to interaction." In practice, this means a world where VR and AR merge into mixed reality - and this is something that QinetiQ have been able to explore thanks to a great deal of innovation and investment. 

Helen explains: "I think you're going to start to find it difficult to distinguish between AR and VR, which is why the term XR has been used a lot. What's changed for us is that the investment in systems which are comfortable to wear and don't have the problems of cybersickness, lag, or latency - which the old systems had - are being addressed by commercial organisations for the entertainment business."

So it would appear the mainstream success of devices like the Quest 2 and the PSVR have benefitted defence and security by allowing the sector to essentially ride the wave of their innovation and investment. But XR takes things significantly further than games and gimmicks by introducing situations where the technology works alongside conventional methods rather than replacing them.​

"Blended is always a solution," Helen says, "but there are incidents where you can't practically and safely practice in real life for those environments; and that could be anything from firefighting on a ship, a mass evacuation of people from an area, or handling of hazardous materials."

Of course, the other exciting practicality of XR training is that you can repeat that task again and again until you get it right, without anyone getting hurt. Helen brings up the example of some VR work QinetiQ did for a coal mining company a few years ago: "To set up a staged accident in a coal mine takes days and resetting is incredibly expensive and time-consuming too. Whereas in VR, you literally bring up the next scenario, do another review and off you go again."

Initially, trainees would move from XR to training in a live environment, but the technology available today allows Helen and her team to scan the environment via a smartphone and recreate it virtually. As avatars become more humanlike and environments become more realistic, the physical training environment might be a relic of the past before long.

Growing sophistication in VR

The progression of hardware over the last few years has been truly overwhelming, with VR headsets getting lighter and more comfortable by the iteration, without sacrificing resolution and power. This is always going to benefit the end-user. The area that's not spoken about as readily, however, is the software driving these headsets.

Helen explains: "There's a big push coming from large companies such as Meta and Epic Games that are creating really usable platforms for you to generate whatever it is that you want to experience in virtual reality."

The space that Helen feels needs a little more attention, however, is haptics. Because while it's wonderful to put yourself in a virtual situation, if you can't feel what's going on there is always going to be an element of cognitive dissonance. This is particularly true when it comes to utilising VR training in the medical field and in situations where sensitive materials need to be handled.

"Where there are intricate procedures that involve your hands there have to be advances in haptics to make that happen," Helen says. "And then in mixed reality when you're starting to touch and feel the objects and then seeing them in the virtual world, you'll be wearing gloves. And if you look at the gloves on the market at the moment, a lot of them are very cumbersome." 

Before we can truly step out of the physical space then, it would seem a greater investment is required not just in how things look and sound in VR but how they feel too. We might be a generation or two behind in that regard, but as VR becomes more widely adopted, haptics will surely follow.

Helen feels that the future of mixed reality will be a natural part of our society but haptics will need to become standardised first. "When you're touching and interacting with your interface, whether that's the cockpit of an aircraft, handling a weapon or whether it's an operational room within command and control. It's got to be completely immersive, otherwise, the bubble bursts and the spell is broken, so to speak."

Enabling a digital vision

Besides a greater investment in the development of haptics, another way the existing XR model needs to change, according to Helen, is that access to entry needs to be streamlined. Helen explains: "In the training that we're doing at the moment we have a bunch of headsets that we send around the country that people use and connect to whether they're at home, in the office or on-site." 

In the future, however, she believes in a service model that will allow people to pick up their headsets from a repository, use them for whatever the intended purpose is and then return them. "It could be that one day you have a combat medic using it and the next day you have a helicopter pilot using it," she says, "because really, the device itself should become like a mobile phone - something that's available to you whenever you need it." 

We're not there quite yet, she admits, but the "Netflix of VR training" is certainly not as far-fetched a concept as it might once have been. One thing that Helen, her team and the rest of the sector will need to contend with before this dream can become a reality, however, is the many barriers to adoption that still exist.

"The biggest challenge that we have is the adoption of AR and VR. Something really has to change to enable that kind of digital vision,"Helen says. "What I do see is that younger people appear to have an innate understanding of VR, whereas many older users still feel a little uncomfortable about the act of physically putting on a VR headset. So finding a way to break through that discomfort is going to be key."

Another barrier she mentions is that people are very enthusiastic about using the technology for training, but when it comes to operations they're a little more hesitant. "We've got to think beyond the conventional skills of engineering and human science," Helen argues, "and bring people from the artistic professions, because they're able to create these environments and tell these stories that become very effective operational vignettes. So that's really interesting, because you wouldn't traditionally be thinking, in defence, that we'd be looking towards art, but actually, in this space, there's a very strong creative requirement."

As for widespread adoption? Helen believes it's only a matter of time. "We just need to help people understand the potential," she says. "Imagine that you're able to put people together into an environment from wherever they are in the world, and manipulate data and show people the pattern of life evolving right before their eyes. That you can even take people to a dataset in 3D and literally pull and play with the data and explore it." Why wouldn't you want to be involved in that?

User-centric design

As a trained psychologist, everything for Helen is about user-centric design. Because technology can be used but not designed with the person in mind. "For us," she says, "it's not just the use case, but it's also about how the person is going to interact with that environment and making sure that whatever they use it for, it works for them. So I have a passion to ensure that it's inclusive and will represent diversity." 

One of the things that QinetiQ as a whole is keen on here is ensuring that it doesn't work only for the individual but for the whole. And Helen thinks everyone in the sector has a responsibility to ensure that diversity and inclusion are built into the foundations of this technology.

"We genuinely believe that XR offers a larger opportunity than training," Helen says. "The potential is almost endless when you factor in human/machine team, AI and machine learning and such a massive amount of data coming in. The end product is going to be something that can be used from the conceptual stage of a project all the way up to the operational stage."

"All of these spaces," she thinks, "will be extremely disruptive and innovative. That's the art of the possible and I think that once we understand the benefit through agile design, that will really fly beyond the use of virtual reality and augmented reality."