I look to my left and see a sorrowful parent sitting on the curb, comforting his daughter. I look to my right, and I see notes of sympathy among many flowers. Around me, I hear people murmuring respects and singing in French. I’m in the middle of a vigil in the streets of Paris, a week after last month’s tragic shooting.
Except I’m sitting at my desk in New York—looking at it all in virtual reality.
What I’m watching is a piece of VR journalism from the New York Times. It’s a new realm of storytelling that the Times is determined to trailblaze: Last month, the 164-year-old paper debuted its very first piece of virtual reality news in an 11-minute long feature called “The Displaced.” In it, you follow the lives of three of the 30 million children driven from their homes in the global refugee crisis. “Vigils in Paris” followed days later.
Today, Times rolls out its latest VR release: For the NYT magazine’s annual “Great Performers” issue, ten Hollywood stars (including Michael Fassbender, Charlize Theron, and Benicio del Toro) recreate great flight scenes from cinema in a three-minute film called “Take Flight.” Simply strap on your VR viewer and look around the room—on screen, you’ll see celebrities floating in the night sky high above city lights, including legendary comedienne Lily Tomlin and Straight Outta Compton’s Jason Mitchell. The goal was to prove that VR can be used in a variety of news capabilities, from humanitarian issues to entertainment.
Earlier this year, Graham Roberts, senior graphics editor at the Times, was inspired to help launch the paper’s involvement in VR after attending SIGGRAPH, the annual computer graphics conference, in 2014. It was attended by huge production companies, like Pixar and DreamWorks. “The whole conference was about VR,” Roberts says. He came back to New York set on using virtual reality in journalism. Fast forward to last month, when the paper released the NYT VR smartphone app and mailed Google Cardboard viewers to Sunday subscribers for free. (NYT VR films are also available on desktop, as well, and can be viewed using YouTube’s new 360-degree videos.)
In some ways, VR is a response to the minute-long-max viral nuggets that have kept online readers’ attention spans fickle and fleeting. If tweets and Vines are shots of tequila, VR is an aged brandy meant to be sipped and savored.
“You can’t really think of VR in the traditional lens,” Roberts says. “[Viewers] need to orient themselves, and you need to give them time to look around, and explore people’s expressions and artifacts.”
Meanwhile, around the same time, Sam Dolnick, a mobile editor in the newsroom, was working with Times magazine editor-in-chief Jake Silverstein, trying to think of ways to incorporate VR in their work. He says delivering the headsets with the paper was a fantasy at first—but it was one that actually came true. Since VR is an emerging technology, it runs the risk of only being used by early adopters and techies. But the key is getting it into everyone’s hands, making it the new normal.
“I’m convinced that if we hadn’t done the Cardboard piece of it, it would’ve fizzled,” Dolnick says.
It’s hard to deny the emotional impact that absorbing a news story in virtual reality has. In “The Displaced,” you’re meeting real kids who not only explain in their own words and voices the tragedy of their situation, but you’re actually in the rubble of their classrooms with them, and you’re actually in South Sudanese fields with the families, literally looking up into the sky and watching the aid packages parachute down from the planes above. You’re actually surrounded by real humans who are overjoyed, talking to each other, scrambling in the grass at your ankles to open up the dropped boxes of supplies.
In short, it really is like you’re there.
Before you actually try it yourself, it might be easy to quickly dismiss VR as a gimmick, as some kind of whiz-bang flavor-of-the-month technology. The Times wants to avoid this at all costs.
“We wanted the journalism to be front and center,” Dolnick says. That’s why they picked a story with such gravitas as the refugee crises in Ukraine, Lebanon, and South Sudan to be the debut piece for NYT VR.
“If we’re just making cool toys, that’s the trap,” says filmmaker Ben Solomon. “The biggest pitfall of this thing is for it to just be a toy. We cannot let this become a toy.”
He says that, among some filmmakers, 2D movies are already becoming outdated—some go so far as to dub them “flatties.”
Solomon’s one of the two photographers (the other being Leslye Davis) who were in Paris mere hours after the attacks on November 13. It was a test: While the Times had already tackled long VR features with “The Displaced,” which allowed months of preparation, the Paris attacks were the Times’ first opportunity to use VR in a breaking news situation. It was also the first entirely in-house VR production for the team. For “The Displaced,” they teamed with a VR company called Vrse.
The technology for shooting VR is still pretty crude, at this point in its lifespan. Solomon and Davis used six GoPro cameras strapped to a custom rig that shot in 360 degrees. With VR, Solomon says, “Everything’s at its birth.”
Directing for VR is also totally different. For 2D films, directors use cuts and edits to directly guide how the viewer experiences the film. But with VR, they completely relinquish that control. The viewer is free to look at whatever they want, whenever they want, and for however long they want.
Since it’s filmed in 360 degrees, the filmmaker can’t be behind the lens—that’d mean they’d be in the shot. So Solomon and Davis had to just set the makeshift rig in the middle of a crowd, run off, observe from afar, and hope for the best.
So what needs improvement in VR, from a technical perspective—besides avoiding making viewers nauseated? (Which is a genuine concern.)
“Everything’s a hack,” explains Solomon, who’s accustomed to shooting in war zones in Africa and the Middle East. “I work in pretty remote places in really difficult scenarios. I want a [VR] camera that’s durable. [Right now] it’s just a bunch of little cameras stuck together.”
Plus, post-production is an incredibly tedious process. Solomon and Davis had to email massive videos of up to seven gigabytes, collected by six separate cameras, back to the Manhattan newsroom. There, each individual frame was painstakingly stitched together by the video team so that all 360 degrees of footage appeared smooth and seamless.
Roberts says that positional audio is next—that’s when audio “moves” with your head. So if you look somewhere, the directions where the sounds come from will change, just as they would in real life. Right now, it’s just like audio in a movie. Solomon thinks that, in as little as six months, the technology will be better: There might be cameras especially designed for shooting films for VR, for example.
And it’s crucial that the technology keeps pace with the team’s editorial ambitions, especially since the Times is far from the only news outlet that’s looking to use virtual reality as a new storytelling tool.
The Associated Press and ABC News, two other decades-old outlets, have also started to use VR to produce content. More news organizations will want to use virtual reality to tell more boundary-pushing stories faster and in more impactful ways. To do that, the hardware and the software have to be more powerful. For example, this week, ABC took viewers inside a 3D tour of North Korea—it’s a prime example of a big “get” whose immersive quality makes it a great sell for virtual reality.
While he declined to give specific numbers, Dolnick says that NYT VR went on to become the most successful app launch the newspaper has ever seen.
“It’s really interesting to set the rules as we go,” Solomon says. “The ground floor is being made right now.”
All images courtesy the New York Times
Contact the author at bryan@gizmodo.com, or follow him on Twitter.