The poor Russian man above is riding a roller coaster in VR, and because VR is shockingly realistic to your vestibular system, he’s reacting to every climb, turn, and descent. He’s also wearing an Oculus Rift DK1. That’s the first major consumer VR headset, released back in 2013. It’s an extraordinary piece of gear, that, according to Oculus co-founder and former Chief Engineer Jack McCauley, is on par with today’s most popular headsets—the mobile kind like the Google Daydream and Samsung Gear VR. Unfortunately, both the DK1 and current mobile headset have one crucial problem: They’re lacking in what VR engineers like to call “localization.”

A far cry from how the term is used with software developers (it’s the procedure employed to translate software for specific countries and locales), VR localization is about tracking the body in a virtual space. Some current systems do it pretty well, tracking not only where your head turns and how fast it turns, but where you exist within the confines of the virtual experience. If shoved, like the terrified man above, you lurch forward in the virtual space too.

Screaming VR Man doesn’t have that luxury. When he’s pushed, his eyes don’t see a corresponding movement. Instead it’s like he’s strapped down, unable to move while being violently (from his perception) assaulted. VR is facing an uphill climb because no one has truly, perfectly unlocked localization, directly mimicking the relationship between the eyes and vestibular system in the real world.

“When I was working at Oculus, I took it upon myself to do research into the kind of affects of VR,” McCauley told Gizmodo. He was one of the few. The rest of the company, passionate gamers eager to realize the dreams promised by Star Trek’s Holodeck and William Gibson’s cyberpunk novels, were focused on creating a cool product that wowed the senses. The potential nausea, vomiting, sweating, fatigue, and disorientation were concerns but not a priority.

McCauley, who, as Chief Engineer on the DK1 and DK2, spent a lot of time trying to figure out the relationship between VR, our eyes, and our vestibular systems, noted that there still isn’t enough research being done outside the military.

“I don’t think we understand enough about how the brain works and the vestibular and ocular system,” McCauley said. Their relationship with VR remains a mystery, and until it’s solved there’s always going to be a threat to our bodies. One wrong slip off the stool or push by a friend and you could wind up like that poor Russian man, screaming into the virtual abyss.

As McCauley notes, different audiences have very different expectations when slipping on a VR headset: Non-gamers can better tolerate low frame rates and artifacting, while regular gamers won’t. Gamers make up the bulk of the people currently developing VR, and everything from mobile headsets for your phone to the $800 ones you plug into your computer are being tailored to them. As someone with three different consoles, too many hours thrown into Overwatch, and a fond memory of Virtual Mario, I was one of the gamers harboring an unconscious need for speed.

The first time I tried VR, I was using an old gaming laptop and an Oculus Rift DK2. I didn’t pay attention to minimum spec requirements for a PC. I didn’t think of how a slower PC might dramatically change the VR experience. In my mind it would be as simple as mirroring the screen on the laptop to the two in the headset—the chunky behemoth with its mobile video card would be enough.

Then I tried playing Alien: Isolation. The computer’s fans hummed angrily as it struggled to output video to the headset. It was having to power not one, but two 1080p displays inside the headset, while immediately rendering a whole new world with every quick turn of my head. The alien’s tail swished across the screen at 60 frames per second. That’s more than adequate when you’re sitting three feet back, but the headset was trying to replicate the real world from only a few inches away. That demands, for a gamer, at least 90 frames per second.

When the refresh rate goes below that, VR sickness can set in, and it’s directly tied to how often you move your head. While a high refresh rate might be coveted by gamers, regular joes will find it wanting, particularly if they plan to watch any live action content on it. That’s because all the content we watch on our phones and TVs is coming in at around 24 to 30 frames per second. It gives content, especially content in motion, a pleasant “cinematic” blur that’s impossible to replicate with current VR headsets. Increase that frame rate and actors look like goofy jackasses in a cheap soap opera.

With VR, we’re going to have to adjust our taste and reconfigure what our brains immediately consider “good” cinematography. Filmmakers are involved in this as well. Ang Lee shot this year’s Billy Lyn’s Long Half Time Walk in 120 fps, James Cameron is in talks to shoot the next Avatar in anything from 48 to 240 fps, and Peter Jackson famously got a critical trouncing for shooting The Hobbit in 48 frames per second. Unfortunately, audiences and critics don’t actually like watching the movies at these speeds, except as an oddity.

Images are so crisp and clear in higher frames per second, which is great for exploring a video game, but it makes live action content feel like it’s being performed on a stage. Translating that stage to the equivalent of a 105-inch 360-degree screen demands a whole other level of adjustment from audiences.

Yet it won’t just be eyes and tastes that need to adjust to make VR more viable. It’s the mind too.

Virtual reality sits on the cusp now of what is real and what we perceive to be real. It’s the uncanny valley writ large. An all-together disturbing experience too real for our bodies to ignore, but too fake for our minds to believe.

But things are changing. Back when the first full flight simulators for pilots went online, researchers noticed a problem. Pilots who had logged hours in an actual airplane were getting sick when flying in a sim, while pilots who had logged hours in a sim would get sick when they flew a plane. It’s called simulator sickness, is virtually identical to VR motion sickness, and might be the most profound problem for VR to overcome.

Our bodies are so incredibly in tune with our surroundings that they know, on a neurological level, when something is off. That’s what is at the core of simulator sickness. The pilots were so attuned to one particular extraordinary experience that when they moved to a similar, but just different enough one, they became ill.

Even if you do overcome simulator sickness, you could be in trouble when you return to the real world. “There seems to be some kind of training going on in your head,” McCauley told Gizmodo. If you spend enough time in VR your body could retrain itself. The real world would become the nauseating one.

The current VR market is being driven by gamers. Palmer Luckey, Jack McCauley, and their fellow Oculus founders developed the headset and transformed the VR industry into what it is today because they were enthusiastic gamers ready to drag the world into the virtual promised land.

The only problem is that this virtual promised land isn’t as ready as gamers would like it to be. It doesn’t matter how rarely you get motion sickness in the real world, the virtual one is an inescapable assault on the human body. Its successes come from tricking the mind and confounding the eyes. But our bodies can perceive the unnatural state of virtual worlds, and when they do they try to protect us, manifesting a feeling of wrongness that makes us feel sick. The problem lies in the VR technology itself. It’s just not good enough to make our bodies believe.