Canadian CG studio Spin took on a raft of tasks, including creating flocks of demonic winged assailants, for the big-screen debut of computer game antihero Max Payne.
Video games, it seems, are Hollywood’s new muse. From Lara Croft in Tomb Raider, through the notorious Postal and disappointing Doom, and on to up-coming projects based on the likes of Gears Of War, game-to-film translations are proving a box office draw, even if they’re not too beloved of film critics.
It’s a new slant on an established formula. Previously, some video games were licensed merchandise launched to support a cinematic release. Yet with interactive entertainment titles outgunning cinema on the takings front, Hollywood has been taking a new interest in games.
And this time, it’s not as a cash-in, but as the source of original material. Max Payne is no exception. Originally a third-person shooter video game developed by Finnish outfit 3D Realms and published by Gathering of Developers in 2001, it was later published by Rockstar Games.
Renowned as one of the first games to introduce ‘bullet time’ – as made famous by The Matrix films – the game met with critical success.
The film adaptation puts Mark Wahlberg in the title role, and took nearly $18m (about £12m) at the box office in its first week. It was also a playground for visual-effects house Spin, which provided the bulk of the film’s effects and CG work.
Awarded the picture after seeing off competition from 11 rivals, Spin – a Canadian visual-effects house with facilities in Toronto, Vancouver, Atlanta and a production office in LA – was also awarded all the creature shots following its work on the big creature feature Outlander.
This gave Spin the chance to infuse the feature film with some of its own creative style. “When you do creature shots, you have to be involved in the creative direction of the film,” says Jeff Campbell, VFX supervisor at Spin.
“We’re responsible for the performance of a lead character in the movie. It’s very rewarding work. We were getting a client brief as far as the general intent of the shot, but it was up to us to come up with a cool performance that [director] John Moore liked.
“As with every project, I wanted to maintain an artistic integrity that reflected a unique, surreal insight into the essence of the movie. I like creating shots that have longevity.”
The movie features several standout VFX character shots, including some featuring fully-CG demons called Valkyries. “Creatively, I want the Valkyries to have personality rather than just motion,” says Campbell.
With a schedule of just 15 weeks for postproduction, Spin launched into creating the CG demon right from the off: with the CG model incorporating feathers and complex rigging, render times were potentially time-consuming.
Initial research included bird references for texture, shape and animation studies, with detailed drawings of the demon body parts from various angles supplied to help start the build from scratch.
With the model of the Valkyrie modelled in ZBrush and 3DS Max 2008 (see right), work turned to rigging and animating. The creature demanded a comprehensive approach, as it needed to simulate fl ight and other extreme poses. The creature also had several difference variants, so the rigs had to be scripted, and the body rigged separately from the wings.
The body rig was based on a regular biped system with reversed feet, but the wings proved a challenge. With so many feathers, a rig had to be produced that allowed them to move realistically, yet not be so complex it would be impossible to control by Spin’s artists.
A simple set of controls was added so that animators could swiftly adjust wing shapes without having to animate each feather – although deeper control at feather-level remained possible.
A further rig, simulating collisions and the dynamic effects of wind and turbulence on the features, was added.
“Eventually, we developed a two-part animation and dynamic rig with the aid of the Maya Hair system, and combined them both so that the artists can blend in or out the dynamic effects or let the dynamic systems take over completely,” explains Spin’s lead rigger Glen Chang.
“In the shot where the Valkyrie pulls Owen out of the apartment in super-slow motion, the animator had total control of the rig and was able to achieve the perfect silhouette by animating each feather manually.
"Whereas in the shot where the whole ceiling tears off into a fiery sky above Max Payne, the animator just let the dynamic system take over all the feather animations.”
Once modelled and rigged, the task of adding the CG Valkyries to the footage was handled almost exclusively by tracking software Boujou, with some manual tracking handled in Maya.
“Tracking for Max Payne was generally easier than other projects we’ve worked on because the CG demons rarely, if ever, touched the ground,” says Spin layout artist Phil Dakin.
“When in contact with the ground or other objects in the plate, CG objects can slip if the track isn’t accurate or the geometry isn’t placed correctly in 3D space.”
Dakin says the noir style of the movie presented more significant tracking challenges. CG elements such as shot extensions and matte paintings, coupled with the overall dark, brooding nature of the footage, added a layer of difficulty.
“The dark sets meant that shutters were open longer, so any camera move is sure to contain motion blur, making it more difficult to track,” says Dakin.
“Many shots had changes in set lighting that can also throw off the tracking software – in particular the turntable shot of Max as the drug vision sets in and the roof explodes outward to reveal a dramatic sky swarming with demons.
“Fortunately for this shot, illuminated tracking markers were placed around the set, without which tracking would have been much more difficult,” he continues.
“Once we had a perfectly circular motion path for the camera, we knew it was correct, since the real camera was on a circular rail.”
Another shot that proved particularly thorny is one where a character is pulled through a window by a Valkyrie. The camera follows the character through the window.
This difficult shot was made easier thanks to a raft of contrasting objects in the interior shots for the tracking software to latch onto, and markers placed on the green screen exterior scene.
The CG demon was then tracked onto the scene, with its position based entirely on the position of the character’s shoulders as he is pulled outside by the demon’s talons. Compositing the elements began in Autodesk’s Inferno suite.
“As an artist, I use Autodesk Inferno to develop looks and to build certain elements to pass off to the compositors. Its high-speed interactivity is ideal for trying out various elements to see what sticks,” Jeff Campbell explains.
“I can also work on a timeline of a sequence and swap out shots instantly to check continuity all at film resolution. Tools like this are essential for problem-solving and keeping to a tight post deadline.”
Campbell adds that, as well as producing demanding, high-end photorealistic final results, Inferno allows him to give his characters subtleties, like muscle dynamics and feather dynamics, that the viewer doesn’t see but “if they weren’t there, you’d think there’s something wrong.”
Spin’s work wasn’t limited to creatures and atmospheric CG effects: it also forged the CG matte backdrops elements to the movie.
Spin’s approach to the city CG elements varied according to the nature of the camera move on any particular plate – with some requiring a 3D model build, and others needing large-scale, layered 2D mattes that were later used in Eyeon Fusion’s 3D environment by Spin’s compositing department.
Setting the scene
“Large, multilayer 3D mattes started off as concept paintings that, after client approval, served as modelling, texturing and lighting guides,” says lead matte painter Bojan Zoric.
“Once the camera tracking and scene setup were completed, the environment layers were laid out, either using finished models or stand-in geometry.
"Modelling was most intensive when it came to the detailed foreground elements of city sets, where camera moves would pick up the convincing complexity of detail in the edges of buildings and props.
“The mid-ground models contained less detail and only the main shapes were blocked in, while background structures were comprised of primary shapes or layered projection planes,” continues Zoric.
“The texturing process of the builds relied on camera projections and UV texturing using Autodesk Maya with clean-up being done using Maxon Bodypaint.”
Depending on the nature of the camera move and the number of layers, the team had to set up several projections, with some projectors handling multiple images.
Background layers especially followed this remit, where building shapes were defined by detailed alpha masks. Projection images based on the original concept artwork were painted using Adobe Photoshop, and lighting, shadow and specular detail included as part of the texture painting.
The team sped up rendering times by using simple, ambient shaders from most of the layers in the colour pass. Extra layers were then added to inject more life into the scene, including particle-based snow, embers, steam and a drifting sky.
The entire VFX roster for Max Payne meant that while the movie was based on interactive entertainment, the project itself was no game, and posed a series of challenges.
“Getting the right look for the wings [of the Valkyries] was a challenge,” admits Cambell. “It took some time to build a shader to get that sheen you see on bird’s wings. Also, with wings you need opacity passes in order to let light through the feathers, which tended to be render-intensive.
"The specularity on the body was also always tweaked. At first, [director] John Moore wanted a wet look but that made the skin look too plastic. Our pipeline outputs the Open EXR file format, which contained 64 channels to allow us maximum control over every imaginable part of the Valkyrie in comp.
“I am proud of the look and feel of our shots. They exude emotion, which is lacking in many CG shots today. I’m also proud of the fact we did all this in just 15 weeks without sacrificing creativity.
“I would have loved to do more Valkyrie shots if we had more time. But I guess it’s good to leave the audience wanting more. Part 2, maybe,” concludes Campbell, intriguingly.
Pride of the Valkyries
The Valkyrie creature is a key character in Max Payne. Modelling supervisor Erin Nicholson initially based the creature on the physical reference of the on-set actor, Mako Hindy, and the prosthetics that he wore as the demon on the set.
“I went to the set and took reference pictures of Mako in his prosthetics makeup,” says Nicholson. These photos were then used as a template to create the model in 3DS Max, with the UV layout created in Unfold 3D.
“The wings were modelled based on a concept by Spin’s lead matte painter Bojan Zoric. Placing the wing feathers was a little tricky and all of the larger ones were done individually or with path constraints. The 3D sculpting of the demon body was done in ZBrush 3.1.”
Using the high-contrast inked designs, Nicholson modified some of them into a texture template, and then used the textures as a mask in ZBrush 3.1 and inflated unmasked areas, sculpting the remaining detail on top.
Nicholson says the “template and inflate” method saved time, and acted as a base to add finer details. “Having worked with ZBrush before, there weren’t any big technical challenges,” says Nicholson.
“It’s a pretty straightforward workflow once you’re comfortable with it. ZBrush’s HD geometry did come in handy, as it was able to handle models that other 3D sculpting packages couldn’t at that time. I just had to increase the contrast of the 32-bit ZBrush displacements by setting the displacement tool intensity value to 512 before extracting the maps so that the rendering software would work out the displacements properly. Overall, it was a fun and relaxed creative experience.”
At one point in the film, Max Payne consumes two phials of a strong drug to warm up after jumping into an icy river; the snowflakes become embers and he experiences visions of Valkyries as a side effect.
City of ember
Spin used Maya’s particle system to maximum effect during a transformation scene that sees snowflakes morph into burning embers.
“The snow was simulated as a Maya particle system, with the wind and drifting motion guided by Maya’s turbulence and drag fields, as well as some MEL expressions to add variation to each particle’s velocity,” says lead FX technical director Tim Sibley.
He continues: “Maya’s particle collision event editor was used to trigger the birth of the ember particles, upon collision with proxy geometry ceated to match the set and Mark Wahlberg’s movement within the scene.”
MEL scripting of the particle’s LifespanPP attribute were used to selectively transform the remaining snow particles into embers over the course of a few seconds.
The particles were cached to a series of .PDB files and the rendering was handled via Mayaman and Sitex Graphic’s AIR, which works alongside Pixar’s proprietary software Renderman.
The snow was shaded via sprite images, and because of the monochromatic output, the RGB channels were each used to pass through additional depth information to assist in the compositing process. The embers were rendered via an AIR streak particle type, with procedural texturing to vary the look of each ember.”
Spin was also responsible for one of the films’ key scenes – a hallucination sequence in which the ceiling of a room Mark Wahlberg is in is torn apart by fire. Tim Sibley, Spin’s lead FX TD explains how the scene was created.
The first step was to obtain a steady 3D track of the footage. A large practical light had been used in filming to simulate the interactive light caused by the fiery sky later in the shot; this meant the team had to create a CG ceiling even before the disintegration event occurs.
"The ceiling was modelled in Maya matching set measurements and rendered using mental ray with lighting that was carefully animated to match the changing illumination from the plate photography.
“The destruction effect itself was created by carefully pre-breaking the individual geometry elements of the ceiling tiles and framing, and then exporting that information into RealFlow,” explains Sibley.
“Realflow’s rigid body simulator was used to tear the ceiling apart, with different sections of debris becoming active at different times, and a variety of vortex and turbulence forces driving the motion.”
Realflow was chosen because of its ability to handle large volumes of geometry, while allowing for the number of substep iterations to be carefully controlled.
The simulation, which went through several iterations says Sibley, took around ten hours to compute for the shot. This data was then re-imported as baked animation information into Maya, which allowed for additional keyframe-animated refinements to tweak the results of the simulation.
The team added extra debris using a particle instancing system, and twisted wreckage was inserted along the edge of the ceiling cavity.
The edge was modelled in Maya and then deformed thriough an nCloth simulation to suggest movement caused by swirling winds. The compositing process in Eyeon’s Fusion required a roto of Mark Wahlberg to allow for the CG ceiling-sky to be inserted behind him, explains Sibley.
“Some complex keying work was used to help extract bright light beams from the plate photography, which was overlayed on top of the 3D elements,” he says.
“The Valkyries were animated in Maya and rendered in Air, while the fiery sky was achieved with several layered flame and cloud elements. Some 2D elements were also inserted to sweeten the shot.”
Spin’s lead matte painter on the project, Bojan Zoric, says, “I ended up using the Shave and a Haircut plug-in included in 3DS Max 2008 to brush in some of the more random feathers at the top edges of the wings.”
The Valkyries were constructed through an animation mesh created in 3DS Max; the UV layout was then created in Unfold 3D before the intricate wings were modelled separately.
One of the more challenging shots in terms of compositing shows a live actor pulled backwards through a window by a CG Valkyrie, with the camera following the move throughout.
PROJECT: Max Payne
CLIENT: 20th Century Fox
STUDIO: Spin, www.spinpro.com
SOFTWARE: 2D3 Boujou, Adobe Photoshop CS3, Andersson Syntheyes, Autodesk Inferno, Autodesk Maya 8.5, Eyeon Fusion, Pixologic Zbrush, Sitex Graphic’s AIR, RealFlow 4