The role of GenAI in immersive learning & VR training experiences

There isn’t a single industry that won’t benefit from the sudden widespread accessibility of generative AI, but when it comes to Immersive Learning, that’s especially true. The biggest challenges to creating VR learning experiences aren’t necessarily hardware-related but content-related, and generative AI (GenAI) has a lot to offer in that domain.

VR hardware has become truly state of the art, and with advanced software platforms like Strivr’s, there’s a playbook at hand for rolling out an Immersive Learning program. But the actual content is hard. It can be expensive to generate and complicated to conceive.

To teach the right learning objectives, engage the learner, and customize a quality experience all require a thoughtful approach to content along with the right assets — scenes, characters, animations, voices, and sounds that make up an Immersive Learning scene. It sometimes takes months for companies to put an immersive training module together, because it requires filming scenarios, building bespoke animations, and assembling everything into a cohesive experience.

We already know that AI can help write scripts, generate assets, and create naturalistic role-playing scenarios, but there’s more GenAI can offer to the world of immersive experience. AI and VR aren’t just two independently trendy buzz phrases in business today. There’s a synergy between them that stands to transform Immersive Learning.

How AI is being used in VR learning

It’s not at all a stretch to imagine using GenAI technology for building Immersive Learning experiences. In fact, it’s already being done. Here are three areas of immediate potential, some of which Strivr is already acting on.

• AI-generated scripts: Instructional designers convert a company’s learning objectives into an actual storyboard and script for the Immersive Learning experience. After putting together a solid outline, they can feed it into a GenAI interface trained to write scripts and produce storyboards. Strivr has had a lot of success fueling the content-creation process in this way, and AI can indeed deftly execute a plot based on a cascading list of learning objectives.
• AI-generated sound: Within a VR learning experience, character dialog is used as text input to dynamically generate human-like speech. Many Immersive Learning modules also use ambient noises to scene-set: things like warehouse background sounds or object-interaction sound effects. Existing voiceovers that have been manually recorded can also be edited with AI to fix mistakes in dialog. AI can closely match the voice to make amendments and modify the learning experience.
• Computer graphics: With GenAI, CG artists can automate the generation of generic assets — the background to an office scene, elements like shelves and boxes, and certain characters. GenAI is proving more and more useful for creating a library of elements that content creators can simply add into a scene, assembling them until they form a full package of immersive training content. We’ve already seen plenty of examples of AI being used to create animated avatars that realistically lip-sync to pre-recorded dialog, for instance, during virtual conferences.

With the ability to create scripts, sound, and CG elements using AI, people assembling an Immersive Learning experience can focus on planning and executing the experience instead of micro-managing the components. In this way, GenAI will continue to accelerate the content-creation process so all kinds of L&D organizations can take advantage of Immersive Learning more quickly and economically. As AI gets better and better, the learning experience in a headset will get quicker and easier to produce in a way that’s realistic, interactive, and effective.

The future of AI in immersive learning

AI technology also holds a lot of future promise in terms of VR content creation and insight-gathering, but it might be a year or two away as we wait for the technology to mature.

While ChatGPT has popularized the idea of AI generation, there’s room for improvement in this model. ChatGPT has been trained on internet data, but there are other treasure troves of data for specific AI applications. Companies sitting on VR data have an AI goldmine at their disposal, and a “walled garden” AI model like this would drive results through the roof.

Strivr, in particular, will work toward more and more effective content generation leaning into GenAI — things like assets, avatars, and scripts generated on the fly. Why? Imagine a situation where the learner has mastered one part of the module but struggles with another and needs the ability to practice it in varying ways. The ability for the training to auto-generate multiple additional opportunities to practice a particular skill — for example, identifying hazards — will improve the learner’s retention.

There’s also a potential for AI-powered avatars that can drive personalized, interactive learning experiences. A year from now, an in-headset assistant is plausible: someone who can offer advice in the moment of training. There is a clear path forward in this effort, and AI technology is already pointed in this direction.

The powerful potential of VR data to understand true human reactions

The true value of AI in most applications is not in replacing human experience but in better understanding it. In VR, reactions can be recorded and analyzed to assess certain things like distraction versus attention to external stimuli, events, or conversations. By applying AI to Immersive Learning, L&D organizations can gather data on how individual and collective learners respond to the experience in order to customize it and provide continuously better training.

It’s not just a matter of capitalizing on data, but of collecting the right types of data in the first place by creating Immersive Learning experiences that will result in certain types of valuable data. This takes an Immersive Learning expert that knows what kind of data counts, which is where a partner like Strivr comes in. Strivr currently has the largest Immersive Learning data set in the world — over 1 million learners have been trained using VR via Strivr.

Balancing power with responsibility

With great power comes great responsibility, as they say, and although there’s a hot debate over who actually said this, we can all agree it pertains well to the arena of AI.

AI will soon become less of a novelty and more of a reliable, trustworthy integration into everyday work. Strivr is committed to balancing innovation, responsibility, and a user-centric approach in our adoption and championing of AI.

GenAI, in particular, has transformative potential to revolutionize the learning landscape. AI is already being used to improve quality and reduce production costs in immersive VR experiences. Soon, with GenAI, we’ll be able to tap into rich troves of data and drive on-the-fly, personalized learning experiences. But the real gold that lies in AI is to see what’s possible that we haven’t even thought of yet.