How To Build an Animated Instructor for VR Learning Applications

Recent breakthroughs in virtual reality and generative AI are transforming the landscape of education. Animated virtual instructors are emerging as essential guides in training simulations by anchoring learners’ attention, delivering engaging voice instruction, and providing an intuitive bridge for interaction within complex VR environments.

In this talk, we recount our journey of developing “Celly,” a cell-inspired virtual instructor crafted for hands-on skills training in a virtual biosciences laboratory. We delve into the core design principles and technical challenges that shaped Celly’s creation, from the art of character modeling aimed at achieving a realistic yet welcoming presence to the implementation of real-time lip-sync and natural eye movements that bring her to life. We also explore innovative solutions for integrating dynamic voice synthesis alongside pre-baked lines, as well as custom-made navigation strategies that allow Celly to effortlessly traverse three-dimensional space. Additionally, we discuss how voice communication transforms Celly into an active source of information and a guide for internal navigation. By sharing our design strategies, technical workflows, and user insights, we aim to inspire new approaches in building AI-driven VR characters to transform education from a passive routine into a truly interactive adventure.