There isn't even a consumer-ready virtual reality headset on the market yet, but that isn't stopping a few filmmakers from thinking it's the future. One of the pioneers in the area is Condition One's Zero Point--a film that's as much a discussion of VR's potential as it is a showcase for new technology.
According to Condition One founder Danfung Dennis I was the first person outside the company to sit through the full twenty-minute version of Zero Point, which had me walking through the Electronic Entertainment Expo in Los Angeles, cowering in the midst of a military training exercise (with a full 360-degree image and head-tracking), and flying off a cliff attached to a drone.
I caught up with Dennis to ask him about some of the challenges associated with VR filmmaking – and they are numerous – to get an idea of what this new medium will look like five or ten years down the line. Read on for details on binaural microphones, sickness-inducing escalators, and the limitations of capturing a GB of data per second.
HD: I had a hard time telling on that pair of headphones – when it was the military training, was the audio actually positional?
DD: Yeah, we're recording with binaural microphones. We think audio is essential for immersion and creating a sense of presence, so we've been experimenting with different types of binaural microphones.
In that specific Marines scene I had some binaural mics in my ear canal--one in each ear canal. It uses the shape of your ears, your head as a stereo source to create that positional audio. So you hear gunshots coming from behind you, you hear people shouting from different directions. It's actually quite subtle, but it brings the level of immersion up a huge amount. Most people don't even pick up on it, but it adds so much.
HD: A lot of the footage in here right now cuts off around 180 degrees. Were those filmed with earlier tech?
DD: Yeah, we've been working on this for a couple of years already so our earlier camera systems were only 180 degrees. We wanted to get first the 180 feeling right, and then we moved to stereo and then to 360, and we still have to figure out the tops and bottoms and filling in. As we've been filming our camera systems have been getting better and better, and will continue to get better and we'll have coverage of the entire 360 degree scene.
HD: That hang-gliding moment...I assume it was a hang-glider. [Referring to the part where the camera flies off a cliff]
DD: It was actually a small little remote helicopter... drone that we attached the camera to.
HD: That made me sweat. I was like "Oh shit".
DD:I know, yeah. We attached the camera to a little drone and flew it off the edge of this cliff. We're really interested in drone technology. I think it can move the camera through space in ways that we could never do before, and as a cinematographer and a photographer, we've always been limited by the physical limitations of the rigs we can build and move. With drones we can just start flying them off cliffs and do things we couldn't do otherwise.
HD: Before you were, as you said, limited by rigs. Now you're limited by people's ability to keep their stomachs down. I've used the Rift a lot and during the a scene at games convention E3 (below), I started to lean back and shut my eyes [because I felt sick]. I consider my tolerance for the Rift pretty high; we have people in the office who use it for five seconds and say they're done. How do you get past that when it comes to making a feature length film, and also having the physical Rift on your face that long?
DD: Our priority is to make this a comfortable experience. You're actually the first person outside of our company to view the longer version of Zero Point. We're looking for guinea pigs to test it out on because as you said, you develop your VR legs really quickly so you need new testers that have never tried it before.
We know that the length of the experience is going to be shorter. We're probably not going to do a feature-length, 90-minute film anytime soon. It's just too intense and too visceral for people to take. We think shorter length – 15-20 minutes will probably work better. We're still experimenting, seeing as the hardware gets better how much that removes any type of motion sickness or how much is on our side.
We know really smooth camera movements are essential. That E3 scene, that was a handheld scene, shoulder-mounted, the cameraman walking, so even that slight camera bob is enough to throw you. There's some basic rules that we have to adhere to for comfort. Things we could disregard or could be accepted in a normal film or a normal video format, when it fills your entire field of view you need to rethink basics. So we're thinking really smooth, steady, forward-moving shots work best.
Even things like moving at a 45 degree angle – we had this escalator scene at E3, and the sensation of moving down an escalator really threw people, so we took that out. And that was just from testing, we found people who just couldn't handle the escalator scene. It's just going to be a lot of testing to see what works.
HD: You're a cinematographer normally?
DD: I worked as a photographer first. This was like 2005, I started working for the AP and The New York Times and Newsweek. Went to Iraq and Afghanistan, worked there from about 2006-2010, was embedded with the military there. And it was around that time I found that still images weren't really able to convey that experience. It was just this glimpse.
Moved into video, built a camera system on large sensor DSLRs, shot a film called Hell and Back Again (below) – a 90-minute documentary about a Marine – and again, wasn't able to convey that experience with existing technology and existing mediums. That's when I started thinking about immersion and how do I place people into the story. I was coming out of that film and wanting to pull people further into the experience, so I started Condition One and assembled a team of engineers to look at how we could create an experience where you felt like you were inside the video.
We started on mobile, developed on the iPhone and the iPad using the accelerometer and the gyroscope. That was the perfect stepping stone for when the Rift came out--all that technology applied perfectly to this headset, which is essentially a mobile device.
Now we're trying to grapple with all these new challenges. Once you get the video in there, there's all these other things to think about. There's a lot of work ahead of us, but all we really know is none of the traditional rules really work anymore and we're going to have to invent this new language. It's a new medium, there's going to be syntax and grammar that has to be developed. We're at this really early stage – we have some of the basic technology working, now how do we use it to tell a story?
HD: Obviously when you have a film crew you have people standing around, you have lights, you have a boom op – all these things you can't have when you have a 360 degree panoramic view. What do you do?
DD: Even basic elements of production – where do you put the crew? It's so basic, but we had to have everyone melt away. We could fit three people under the camera in that black blind spot, but even then my head was getting in the frame all the time. You can't have lights. Anything you put in the frame is going to show up, so basic elements of production need to be rethought. Who's essential? You need the [camera] operator, the director, and sound. Everyone else melt away.
We're thinking light, small crews are going to work best. Preplanned shots will be the best ones at first. Here we're just kind of shooting different things we thought would work well, but we really had to think about where the attention of the user is going to be. You don't know where they're going to be looking most of the time, but you can guide them--give them visual or audio cues to pull them in a certain way, to get them to look over here. We're starting to figure out what those are, how we can have a transition to a new scene and have them land in the right place. Everything from production to how you create the scenes to how you draw the attention of the user--all of that is being rethought for VR.
Do you think this technology will be best-suited for documentary-type films? I mean, documentaries normally have light crews and you're not spending an hour on lighting to get a shot--you have to take the shot in the moment. Do you see narrative films being made with this tech?
DD: I'm personally interested in a hybrid between the two. We have an advantage in that we can capture the real world. There are CG films that cost hundreds of millions of dollars and that can create anything. We have the advantage that we can just set up our camera at a real-world location and start filming.
But because the limitations of capture--we're capturing a GB per second, we're filling up hard drives like that--you don't have the freedom of a documentary. I filmed over a hundred hours for Hell and Back Again. [With this tech], we had to roll and get what we need and then just cut it right there.
So I think narrative placed into real-world environments that are interesting and people want to go to is going to work best at first, because we can stage the shots and move the camera exactly how we want to move it, but we're not trying to create an entire scene from scratch. So we're looking at different levels of intensity for these hybrid films. Some are really extreme for people who want intense experiences--maybe a horror film shot in a real-life slaughterhouse. That would be disturbing and terrifying.
My preference would probably be a calm walk along the beach or in a forest. Relaxation-type apps. We think we'll have a spectrum of films that give you different types of intensities. We're finding people have different tolerances. People will take off the headset if it's too intense.