When you strap on all of the gear required for a modern, immersive, virtual reality experience, you’re all but completely blind to the real world. But interacting with real world objects can often enhance a virtual experience, so Disney’s researchers came up with a way to let users catch a real ball without leaving a VR world.


Simply catching and throwing a tennis ball doesn’t exactly sound like a thrilling use of virtual reality, not when you can strap into a roller coaster or battle aliens on a far-away world. But imagine the feeling of grabbing an alien’s tentacle when you engage in hand-to-hand combat. That’s the ultimate goal of research like this, adding a tactile feeling to what’s being experienced in a virtual reality simulation.

So how can you make someone who is blind to the real world catch a ball they cannot see? What Matthew Pan and Günter Niemeyer of Disney Research Los Angeles came up with was a predictive system that tracked the motion of a thrown ball in real-time. Using that data, their software is able to show a VR user a virtual recreation of the real ball, its trajectory as it soars through the air, and a target they should reach out to in order to make the catch.

The success of a VR user catching a real-life ball is dependent on the system’s tracking accuracy, and the software’s ability to translate this data into the virtual world. But the results confirm that virtual reality doesn’t have to be someone just standing in a big empty room flailing their arms around. Robots, and other moving objects, could be used as real-world proxies for what’s happening in a virtual experience, giving users something to actually reach out and touch or, potentially, something to more realistically battle.


You could also imagine an evening spent at a virtual reality batting cage where you’re swinging at real pitches, with the satisfying feeling of the bat connecting with the ball. But according to your other senses, you’re actually at the World Series, swinging for the fences in hopes of making the crowd go wild.

[Disney Research]