James Cameron Supercharges 3-D

"Director James Cameron's upcoming "Avatar" must rank as one of the most anticipated film projects in recent memory. His first narrative film since making the No. 1 box office hit of all time, 1997's "Titanic," "Avatar" will be the realization of Cameron's long-held dream of melding digital 3-D stereo with epic bigscreen storytelling. Variety's David S. Cohen conducted this email interview with Cameron; it is the director's most extensive exploration of 3-D to date, however he is keeping specifics about "Avatar" under wraps.

You've worked in 3-D before and have been an evangelist for this technology. We've heard lots of people in the industry talk about the importance of delivering an in-theater experience that goes beyond what people can get in the home. We're seeing that audiences like 3-D and it's becoming a main driver for adoption of digital cinema systems in movie theaters. But speaking strictly as a storyteller and director, what does 3-D add to the creative side of a project?
I believe that Godard got it exactly backwards. Cinema is not truth 24 times a second, it is lies 24 times a second. Actors are pretending to be people they're not, in situations and settings which are completely illusory. Day for night, dry for wet, Vancouver for New York, potato shavings for snow. The building is a thin-walled set, the sunlight is a xenon, and the traffic noise is supplied by the sound designers. It's all illusion, but the prize goes to those who make the fantasy the most real, the most visceral, the most involving. This sensation of truthfulness is vastly enhanced by the stereoscopic illusion. Especially in the types of films which have been my specialty to date, the fantasy experience is served best by a sense of detail and textural reality supporting the narrative moment by moment. The characters, the dialogue, the production design, photography and visual effects must all strive to give the illusion that what you're seeing is really happening, no matter how improbable the situation might be if you stopped to think about it -- a time-traveling cyborg out to change history by killing a waitress, for example. When you see a scene in 3-D, that sense of reality is supercharged. The visual cortex is being cued, at a subliminal but pervasive level, that what is being seen is real. All the films I've done previously could absolutely have benefited from 3-D. So creatively, I see 3-D as a natural extension of my cinematic craft.

A 3-D film immerses you in the scene, with a greatly enhanced sense of physical presence and participation. I believe that a functional-MRI study of brain activity would show that more neurons are actively engaged in processing a 3-D movie than the same film seen in 2-D. When most people think of 3-D films, they think first of the gimmick shots -- objects or characters flying, floating or poking out into the audience. In fact, in a good stereo movie, these shots should be the exception rather than the rule. Watching a stereo movie is looking into an alternate reality through a window. It is intuitive to the film industry that this immersive quality is perfect for action, fantasy, and animation. What's less obvious is that the enhanced sense of presence and realism works in all types of scenes, even intimate dramatic moments. Which is not to say that all films should be made in 3-D, because the returns may not warrant the costs in many cases, but certainly there should be no creative reason why any film could not be shot in 3-D and benefit from it.

When I started down the path of developing the 3-D cameras with Vince Pace in 2000, we were looking for an alternative to the massive film-based cameras I'd used in the past. Two years later, while deep in stereo technology development and production, I had an epiphany: that the digital projectors being proposed to replace 35mm film, could support 3-D perfectly, because of their high frame rates. They could actually display 3-D by projecting left and right eyes sequentially, at crazy high frame rates, which we perceive as simultaneous. So I figured this would mean that a whole new era of 3-D was now possible, and that our humble 3-D efforts would ride to market on the broad back of the digital cinema rollout, which was seen as imminent and inevitable.

It is ironic that half a decade later, the rollout is happening, but largely because it has been catalyzed by 3-D. D-cinema is riding 3-D to market. And that's because audiences are seeing something they like and are demonstrating a willingness to pay more for it. The new 3-D, this stereo renaissance, not only solves all the old problems of bad projection, eyestrain, etc., but it is being used on first-class movies that are on people's must-see lists. These are fundamental changes from what happened with the flash-in-the-pan 3-D craze of the '50s. 3-D is also a chance to rewrite the rules, to raise ticket prices for a tangible reason, for demonstrable value-added.

Quick definition of terms: I say stereo instead of 3-D, because I deal with so many CG artists who are accustomed to using the term "3-D" as a CG term of art. So I use stereo, a shortened form of stereoscopic, instead, so there is no confusion. However, when dealing with the public, I say 3-D, because they know what that means in that context -- they're going to get to wear the glasses and see something really cool.

Are there any myths about 3-D you'd like to dispel?
I sort of hit the myths one by one in the answers to the questions below.

Trailers and TV commercials are important for marketing, and homevideo is a vital revenue stream, yet right now there's no 3-D TV and you can't always count on trailers being seen in 3-D. How do you handle that as the film's director?
All films are made to serve many masters. Every director knows his film will be seen by more people on DVD or network TV on a small screen than in a theater. Does that change the way we direct? Not much. First and foremost the film must be a good movie. It needs to be firing on all eight cylinders whether it is conceived as a 2-D or a 3-D film. As a result, a 3-D film when screened in 2-D, on a screen of any size, should still deliver. The 3-D should always be thought of as a turbocharger, an enhancer, to a work whose raison d'etre is vested in its story, its characters, its style, etc.

In any case, with the number of screens currently available in North America, and certainly for some years to come internationally, it will be necessary to release in 3-D and 2-D day and date. So the film must be fully competitive as a 2-D title as well. Before I decided to make a major movie in 3-D, I had to resolve to my own satisfaction that the 3-D would not degrade in any measurable way the 2-D viewing experience. Could I shoot the same way? Would the camera placement or lighting be compromised? Could I cut as fast? Etc. Only when I had done enough 3-D production and testing to answer these questions was I willing to proceed.

As for 3-D in the home: The only limitation to having stereo viewing in the home is the number of titles currently available. When there is more product, the consumer electronics companies will make monitors and players. The technology exists and is straightforward. Samsung has already shipped 2 million plasma widescreens which can decode an excellent stereo image. There's just no player to hook up to it right now. They may be a little ahead of the curve in future-proofing their monitors, but it indicates how easy it would be for the big electronics companies to get onboard. It should be remembered that good 3-D requires a more immersive relationship between audience and screen. Unless you're willing to sit within 4 feet of a 50" monitor, which all but a few geeks (like me) will not do in a home setting, then you're not going to get the same bang for the buck out of a 3-D movie on a home system as you would in a theater, regardless of whether the resolution of the image is the same. So there may always be a greater distinction between seeing a 3-D movie at home vs. seeing a 2-D movie at home. Which is good. Because 3-D then becomes a technology which will help preserve the health of the theatrical exhibition business in a time when it is besieged.

Do you think it's possible to make a film that is too dependent on 3-D for the economics of today's movie business, and if so, how do you avoid that?
I don't think the economics of 3-D are clear yet, and won't be for a few years. So much depends on the number of screens, and more importantly (ultimately) the number of filmmakers who want to play in this new space, because the success of the 3-D renaissance is going to be content-driven. I think it is a mistake under any circumstances to make a film which is dependent on 3-D for its success, either aesthetically or commercially. The film should not be marketed first and foremost as a 3-D experience. The film should be sold on its merits (cast, story, imagery, etc.) and the consumer should be informed that they can purchase the experience in 2-D or, for a couple extra bucks, in 3-D. It should be like ordering at Starbucks. Lots of choices. If the new media of the last decade has taught us anything, it is that people like choices, and they like control.

WORKING IN 3-D:

How do you shoot differently because of 3-D?
On "Avatar," I have not consciously composed my shots differently for 3-D. I am just using the same style I always do. In fact, after the first couple of weeks, I stopped looking at the shots in 3-D while I was working, even though the digital cameras allow real-time stereo viewing. I had someone else checking them for good stereo as we were shooting, in a small theater we set up near the stage for that purpose. I would get real-time feedback from my "golden-eyes" team in the theater, if a shot needed to be adjusted to increase or decrease the stereospace.

Having said that, I am not above milking a good 3-D moment, as long as it doesn't interrupt the narrative flow. And there are a couple of minor adjustments that need to be made to lighting and camera placement to create a smooth and unobtrusive stereo experience. But once you learn these few tricks, you stop thinking much about them.

In general I found that good lighting was good lighting, and worked quite well in 3-D. Wide lenses are fun in 3-D, but long lenses work well, too. The Fusion cameras can dynamically shift from hypo-stereo, which is to say less than normal interocular distance, (the distance between the left eye and the right eye lenses) for closeups -- to hyper-stereo (wider than normal) for long lens shots where the subject is relatively far away. The new cameras work well on Steadicam, on cranes and dollies, on SpiderCam and Cablecam rigs, and work very well handheld. So all the normal types of shots can be done. I compose the shots on a 2-D monitor, while in the back of my mind I'm imagining it in 3-D. That way I know I'm always making a good 2-D movie as I go along. I also edit in 2-D, for the same reason.

Someone told me that "Citizen Kane" was a great example of how to shoot for 3-D: great depth of field, wide-angle lenses, etc.
I think it's a myth that you want deep focus in 3-D shots. I find the opposite is true. Selective focus, created by working at low f-stops with longer lenses, evolved as a cinematic technique to direct the audience's attention to the character of greatest narrative importance at a given moment. With 3-D, the director needs to lead the audience's eye, not let it roam around the screen to areas which are not converged. So all the usual cinematic techniques of selective focus, separation lighting, composition, etc., that one would use in a 2-D film to direct the eye to the subject of interest, still apply, and are perhaps even more important. We all see the world in 3-D. The difference between really being witness to an event vs. seeing it as a stereo image is that when you're really there, your eye can adjust its convergence as it roves over subjects at different distances. Convergence is the natural toe-in that the eye does to align the left and right eye images of objects at specific planes of depth. In a filmed image, the convergence was baked in at the moment of photography, so you can't adjust it. In order to cut naturally and rapidly from one subject to another, it's necessary for the filmmaker (actually his/her camera team) to put the convergence at the place in the shot where the audience is most likely to look. This sounds complicated but in fact we do it all the time, in every shot, and have since the beginning of cinema. It's called focus. We focus where we think people are most likely to look. So I've found that just slaving the convergence function to the focus works exceedingly well, and makes good stereo a no-brainer on the set.

Every time I watch a movie lately, from "300" to "Atonement," I think how wonderful it would have been if shot in 3-D.

How does that third dimension change or complicate those directing techniques?
Shooting 3-D is more complicated, undeniably, because you're doing all the stuff you normally do (blocking, lighting, performance, etc.) plus dealing with stereospace. From a director's perspective, the camera team should be handling most of this, and the director need only get involved to the extent that they choose to, because they're excited by the new format and tools.

How does working in 3-D change the way you cut a film? The current trend toward very quick cuts, so popular now in action films, seems not to work in 3-D. Or does it?
The new cameras allow complete control over the stereospace. You should think of interocular like volume. You can turn the 3-D up or down, and do it smoothly on the fly during a shot. So if you know you're in a scene which will require very fast cuts, you turn the stereo down (reduce the interocular distance) and you can cut fast and smoothly. The point here is that just because you're making a stereo movie doesn't mean that stereo is the most important thing in every shot or sequence. If you choose to do rapid cutting, then the motion of the subject from shot to shot to shot is more important than the perception of stereospace at that moment in the film. So sacrifice the stereospace and enjoy the fast cutting. Stereo is just another color to paint with, and the new camera tools allow complete control. I think it takes a few frames, maybe the better part of a second, for the eye to properly assimilate the stereospace of a shot. If the shot only lasts 18 frames, you're not getting much value out of the 3-D, so let that drop down in priority below the flow of the motion.

The real issue here is that when you're shooting action photographically (as opposed to CG animation) you can't predict at the moment of shooting exactly how you're going to cut, so it pays to be conservative on the stereospace. In a CG action sequence, you can pump the stereo up a bit more because you can optimize each shot after the scene is cut. The interocular continues to be malleable up to the final render of a CG shot, but it gets baked into a photographic shot the moment you pull the trigger and can't be changed later.

Does directing in 3-D require that the director and producer have a thorough grasp of the technology, or is this something an inexperienced director could mostly delegate to a d.p. and stereographer the way a writer or actor turned director might delegate camera angles and lighting to the d.p.?
Most directors couldn't load a film magazine or balance a Steadicam to save their lives. But that doesn't stop them from using these tools brilliantly. Stereo should be thought of in the same way. A good, experienced camera team which has shot a stereo movie using the new tools should be able to make the stereo as invisible to the director as focus. Meaning, sometimes the director gets asked where they want the focus in a shot, or the director may have an idea before the fact to do something stylized, but generally it just gets taken care of by the camera team. I do believe in the need for a "stereographer" to assist the d.p. This should be an experienced person who watches each and every image as it is laid down, and advises the director and d.p. regarding the stereospace decisions, based on what they're seeing at the moment.

Of course many filmmakers will be drawn to shooting in 3-D because it is fun, new and challenging, and they will meet that challenge by learning the ins and outs themselves, and learning-by-doing how and when to push the envelope. Fortunately, the new 3-D cameras are able to meet their revolutionary performance specs, that no film camera could dream of matching, because they are HD. So that immediate real-time stereo image is there for the filmmaker to experiment with.

And every single director will approach 3-D in their own way, and use it differently. So even though I believe that a standardized methodology is necessary for widespread adoption, that methodology needs to be open to the creativity of the individual filmmaker.


DIRECTING ACTORS:

Last year Variety did an article on how digital capture changes the way actors work (The actors said no reloading means fewer breaks to prepare, much more continuous shooting, more of their process recorded for posterity so they have to have less ego). I've recently talked to the "Beowulf" vfx team, which said performance capture let them shoot very fast, with very little downtime for the cast. They were moving so quickly that the actors had to ask for breaks to work on lines, because they weren't expecting to get to the next scene so soon.
I didn't experience that. We were doing lighting, figuring out shots, moving assets around in the CG environment on production days with actors. This took significantly longer than the smash-and-grab mo-cap techniques used previously. Also, I tend to spend a lot of time on performance, so nobody was complaining about the speed.

Does 3-D also change the way actors work or the way you work with actors, and if so, how?
I made it my mission to keep the 3-D out of the actors' consciousness completely. Most of them forgot we were shooting 3-D, because we did playback on set at a 2-D monitor. Every once in a while one of them would go over to the theater and watch some dailies, and come back wide-eyed. But it really didn't change a thing they were doing on set. As a director, my work with the actors was not affected in the slightest by the 3-D component of the shooting.

As for the lighting and photography, we found that the normal gutsy lighting that I like worked beautifully in 3-D. Every once in a while we would have to make an adjustment to hide or reduce "ghosting" of a bright light in the background. Ghosting is an artifact of projection, not photography, but we decided to mitigate it in the photography to improve the experience in the theater. Hopefully, as projector technology improves, we can forget about that.

Right now, 3-D is pretty much being used for films that have some spectacle in them, whether it's "Journey to the Center of the Earth" or "U2 3D"; nobody's talking about using it for domestic dramas. But there are people wondering whether it will actually enhance the impact of character-driven stories. What are your thoughts on how 3-D changes the experience of watching actors act?
I plan to shoot a small dramatic film in 3-D, just to prove this point, after "Avatar." In "Avatar," there are a number of scenes that are straight dramatic scenes, no action, no effects. They play very well, and in fact seem to be enhanced by the stereo viewing experience. So I think this can work for the full length of a dramatic feature. However, filmmakers and studios will have to weigh the added cost of shooting in 3-D against the increased marketing value for that type of film.

3-D POST AND PROJECTION:

We've only just seen an all-digital pipeline come into being.
I've been doing it since 2001.

What about an all-3-D pipeline?
You don't need to be in 3-D at every step of the way. And as long as your work will be viewed in 2-D as well as 3-D, whether in a hybrid theatrical release or later on DVD, it is probably healthy to do a lot of the work in 2-D along the way. I cut on a normal Avid, and only when the scene is fine-cut do we output left and right eye video tracks to the server in the screening room and check the cut for stereo. Nine times out of 10 we don't change anything for 3-D. I operate most of the shots myself, including the handheld (I defer on the Steadicam shots), and we use 2-D monitors and eyepieces to operate. On-set playback is in 2-D. A shot is judged on the merits of performance, operating, lighting, etc., and not 3-D. I think this is a healthy approach.

Where is the existing pipeline working well and where do things still need to be improved -- or invented -- in 3-D production and post?
3-D post is mature and pretty straightforward. If the material is shot properly, you don't need to do much to "fix it in post." Witness the Hannah Montana concert movie, which was posted in less than three months. The visual effects pipeline could use some good stereo tools, to aid in compositing.

I'm hearing that there are already calls to increase the frame rate to at least 30 fps for digital 3-D because certain camera moves, especially pans, look jumpy in 3-D. I saw that in the Imax 3-D "Beowulf." You've been an advocate for both 3-D and higher frame rates. Have you seen the problem and do you have any thoughts on it?
For three-fourths of a century of 2-D cinema, we have grown accustomed to the strobing effect produced by the 24 frame per second display rate. When we see the same thing in 3-D, it stands out more, not because it is intrinsically worse, but because all other things have gotten better. Suddenly the image looks so real it's like you're standing there in the room with the characters, but when the camera pans, there is this strange motion artifact. It's like you never saw it before, when in fact it's been hiding in plain sight the whole time. Some people call it judder, others strobing. I call it annoying. It's also easily fixed, because the stereo renaissance is enabled by digital cinema, and digital cinema supplies the answer to the strobing problem.

The DLP chip in our current generation of digital projectors can currently run up to 144 frames per second, and they are still being improved. The maximum data rate currently supports stereo at 24 frames per second or 2-D at 48 frames per second. So right now, today, we could be shooting 2-D movies at 48 frames and running them at that speed. This alone would make 2-D movies look astonishingly clear and sharp, at very little extra cost, with equipment that's already installed or being installed.

Increasing the data-handling capacity of the projectors and servers is not a big deal, if there is demand. I've run tests on 48 frame per second stereo and it is stunning. The cameras can do it, the projectors can (with a small modification) do it. So why aren't we doing it, as an industry?

Because people have been asking the wrong question for years. They have been so focused on resolution, and counting pixels and lines, that they have forgotten about frame rate. Perceived resolution = pixels x replacement rate. A 2K image at 48 frames per second looks as sharp as a 4K image at 24 frames per second ... with one fundamental difference: the 4K/24 image will judder miserably during a panning shot, and the 2K/48 won't. Higher pixel counts only preserve motion artifacts like strobing with greater fidelity. They don't solve them at all.

If every single digital theater was perceived by the audience as being equivalent to Imax or Showscan in image quality, which is readily achievable with off-the-shelf technology now, running at higher frame rates, then isn't that the same kind of marketing hook as 3-D itself? Something you can't get at home. An aspect of the film that you can't pirate.

Other than that, for digital 3-D, would you rather see energy going into moving from 2K to 4K, or into moving from 24 fps to 48 or 72 fps, and why?
4K is a concept born in fear. When the studios were looking at converting to digital cinemas, they were afraid of change, and searched for reasons not to do it. One reason they hit upon was that if people were buying HD monitors for the home, with 1920x1080 resolution, and that was virtually the same as the 2K standard being proposed, then why would people go to the cinema? Which ignores the fact that the social situation is entirely different, and that the cinema screen is 100 times larger in area. So they somehow hit on 4K, which people should remember is not twice the amount of picture data, it is four times the data. Meaning servers need to be four times the capacity, as does the delivery pipe to the theater, etc.

But 4K doesn't solve the curse of 24 frames per second. In fact it tends to stand in the way of the solutions to that more fundamental problem. The NBA execs made a bold decision to do the All Star Game 3-D simulcast at 60 frames per second, because they didn't like the judder. The effect of the high-frame-rate 3-D was visually astonishing, a huge crowdpleaser.

I would vastly prefer to see 2K/48 frames per second as a new display standard, than 4K/24 frames per second. This would mean shooting movies at 48 fps, which the digital cameras can easily accommodate. Film cameras can run that fast, but stock costs would go up. However, that could be offset by shooting 3-perf, or even 2-perf, because you'd get the resolution back through the higher display rate. The 48 fps negative or digital master can be skip-printed to generate a 24 fps 35mm DI negative for making release prints, so 48 is the magic number because it remains compatible with the film-based platform which will still be with us for some time, especially internationally. 30 and 60 fps are out for that reason. Anyway the benefit of 30 is not great enough to be worth the effort, especially when 48 is so easy to achieve. SMPTE tests done about 15 years ago showed that above 48 frames the returns diminish dramatically, and 60 fps is overkill. So 48 is the magic number.

Of course, the ideal format is 3-D/2K/48 fps projection. I'd love to have done "Avatar" at 48 frames. But I have to fight these battles one at a time. I'm just happy people are waking up to 3-D.

Maybe on "Avatar 2."

It's turning out that 3-D that's optimized for one screen size doesn't look right if the screen gets a lot bigger or smaller.* One potential solution would be correction built into the software at the projector, but the people I've spoken to who actually make 3-D movies think that these are creative decisions and different 3-D masters will be needed for different screen sizes. Do you think this is something you would ultimately trust to software or will you need to do it yourself?
*(Specifically, the interocular changes by the same multiplier as the screen size. Double the screen size and the interocular doubles too, and can be so big that it's difficult for the eye to resolve the stereo. On the other hand, cut the screen size in half and the stereo effect flattens out.)
I don't agree with this at all. I think the effect you are describing has more to do with the fact that people tend to sit farther from monitors than they do from cinema screens, when calculated as a ratio of viewer distance to screen width. If you sit close to a good stereo monitor, like the Samsung I demo'ed a few months ago, the stereo effect is the same as a cinema screen. The stereo effect even works on smaller monitors. The advantage of small individual monitors, like laptops, is that they will be available as autostereoscopic displays, meaning no glasses. I've seen demos of these, and the effect is good. The ones I saw just suffered from low frame rates (flicker), but they'll work that out.

I certainly would never change the stereospace of a film to fit different screen sizes. In fact, for photographic films, it can't be changed. The interocular is set at the moment of photography. People will tell you they can fix it later, in post, by changing the convergence, but they are wrong. Convergence does not change stereospace, it only changes the ease with which viewers can fuse a shot after it appears onscreen.

In fact, I would go so far as to say that 10 or 15 years from now, stereo displays will be ubiquitous, from cinemas to open-air advertising, to home screens and down to handheld devices. IPhones will be in stereo. Small displays will especially benefit from stereo because the small size of the screen can be offset by using Z-depth to stack information, which will reduce visual clutter, or conversely increase the density of information held within a single visual field. It may be that eventually all of our news and information, as well as our sports and entertainment, will come to us in stereo.

In the future world shown in "Avatar," all display devices, including handheld devices and even photos, are all in 3-D.

We evolved to see in 3-D for a reason. It made us better hunters, or allowed us to spot and avoid predators. Why wouldn't we want this Darwinian edge in our workplace, in our sports and entertainment, in all our peak visual experiences?

You know what I think."

By David S. Cohen, Variety