It taken countless hours to fill three consecutive films full of the most realistic ape characters ever seen on screen for the revived Planet of the Apes trilogy, but for Weta Digital director Joe Letteri, one of the most accomplished visual effects artists in the industry, it was all worth the effort.
With the trilogy’s final chapter War For the Planet of the Apes now available on home formats, Letteri took a moment to look back on what will inevitably be looked on as one of the most significant achievements in the effects field, and one that will spark pioneering breakthoughs yet to come. And that’s saying something, considering that Letteri and his team at Weta have already changed and re-changed the FX game multiple times over, most notably with their work on Peter Jackson’s Lord of the Rings and Hobbit films and James Cameron’s Avatar.
As he takes a breath before digging in deep to his upcoming projects Alita: Battle Angel, for producer Cameron and director Robert Rodriguez, and a planned series of new Avatar films, Letteri joined CBR for a look at the totality of his Apes experience, the particular magic of Andy Serkis, and what further bar-raising visuals the future might hold.
CBR: When it comes to the Apes trilogy and the scope of it, I'm sure when you started you expected that if the first film was successful, there would be more than one movie -- so what was the vision going in for how your team at Weta was going to approach your end of it?
Joe Letteri: I think the most important part for us was to take the idea that it was an origin story starting in the modern day, and figuring out how to make those chimps look realistic. They have to be indistinguishable from what you could go out and photograph. But then to start bridging that gap towards them getting intelligent -- and especially the ability to speak, because Caesar starts to speak at the end of the first film.
So even then we had to make a few design choices, like giving him a slightly smaller muzzle so he could articulate better when he did have to do that, and then just from an expression aspect, bringing in some of Andy's features, especially around the eyes. Because you're watching Andy's performance and you want to recreate it, and so much is in the eyes.
So what we did is surrounded him with chimps that were completely based on real chimps that we could photograph and study, and then fit Caesar into that with his slightly more unique design and treat that as part of his personality, like he was born with the drug in him, where he started to have a little bit of this intelligence. And that gave us the springboard to give him those unique properties.
And then as the films developed, like in two and three when the other apes had to start showing intelligence, we just made subtle changes between each film. And we hid it in the aging process, right? So we made their muscles slightly smaller so that they too could speak, and made their eyes a little bit more expressive. They still look like the same character, because you're 15 years later and you expect some changes. But it gave us that grounding to be able to show the enhanced intelligence.
Tell me about working with Andy, who cracked the psychology of Caesar for each film, and incorporating his performance into what you guys were trying to do visually. That merger is so seamless in these films.
We spent a long time on the first film working with Andy and working with Terry Notary and working with all of the other Apes actors like Karin Konoval who played Maurice, to really understand ape behavior. Because a lot of it is very physical. Apes have longer arms and shorter legs than we do, so the actors have to learn to move with arm extensions because apes are quadrupedal. And that physical performance has to become the basis of it. Because if you make the apes look completely realistic that they stand up and just walk around normally and you capture that, it looks like a man in a suit. It just looks like what it is, because the performance comes through so faithfully.
So that physicality had to be like the groundwork that you didn't even think about. That's what the basis of it was, and everything else got layered on top of that. So we spent a lot of time working with it interactively to understand that, and to really talk about the psychology of the character and where the arc was going. But by the time we got to the third film, they all knew what they were playing.
So for us, it just became a question of just paying attention to the details -- which is good, because in the first film we were wondering, "Do they look chimp-like enough? Are they moving the right way?" We were focusing on the translation aspect of it. But really, by the time we did the third film, we could really just delve deeper into the artistic side of it, really making sure we got the nuance of the performance. Because that was really important for War.
Was there a new breakthrough each film? Was there a level that you crossed every time you made one of these movies?
Well, the biggest breakthrough was the first one. Just being able to take performance capture out onto the stage, right? Because before that, like when we did, say, Lord of the Rings with Andy, he'd be in there with the actors, but we'd have to take him back to a motion capture stage with all the cameras around and say, "OK, Andy, here's what you did. Can you do it again?" And capture it and put it into the plates. Because performance capture is really a bunch of cameras shining a bunch of lights at the actors to try to see their performance from dozens of different angles so you can reconstruct it. That doesn't play well with the photography, because you're throwing extra light into the scene, and vice versa, the lights that are lighting the scene can confuse the performance capture cameras.
So our big breakthrough was figuring out how to do that in Rise. And once we had that part cracked from the capture aspect of it, it just became more and more about making this transportable anywhere that you wanted. Because [director] Matt [Reeves] had us going out into the woods where the conditions were just really remote and harsh, so we had to just figure out, technically, how do you keep the weather from affecting this? So it's pretty delicate gear, you're trying to capture movement down to a sub-millimeter level -- and it's raining! Like in War, it's snowing and the actors are rolling around, and it's just really physically demanding. So we were able to evolve that side of it through the three films.
So when we call it performance capture everyone thinks of it as, "Well, here's Caesar's performance," but really you're capturing the actor's performance. The other side of it is the translating that and making the character on the screen, and there are a lot of aspects to that. We've gotten better doing muscle simulations, we've gotten better at doing the fur dynamics, the way light scatters in the fur, the way debris accumulates in the fur, rain, snow and all those sorts of things -- better at creating the physicality of the lights in the world. It's called a renderer, which is the software that actually traces what all of the light rays are doing in the scene, and it much more accurately mimics what's happening in the real world.
And then we broke that out completely for War, so it was done 100 percent with this new renderer that’s called Manuka. So there are all these breakthroughs that add to the realism, but they all have to work together, right? If any one part of it falls down, you tend not to believe any of it.
As more and more actors become conversant in this style of acting that's necessary for performance capture, why is Andy still the top of the heap? What is the secret that you see in him approach?
I think, in a word, Andy is fearless. He is totally committed to the character. And to him, the lack of a costume doesn't really matter. In his head, he is Caesar. And he also has a lot of confidence that he knows that we're gonna give him that back at the end of the film. If he sees it, it will be his performance, but it will be Caesar doing it. And I think a lot of actors now that they've seen the process are becoming more comfortable with that.
In the early days Andy had a lot of faith to just let us do that right off the bat with Gollum. But when you work with new actors, they have a little bit of hesitation. "It's not my face, what's gonna come through, is it gonna look like me? Are you basically gonna screw this up?" And now that they've seen enough of the process, that hesitation goes away. So a lot of actors just walk onto the stage and they're comfortable with it now.
And my feeling from talking to a lot of actors is, to them it's like a workshop. They're just in the heads of the characters, the characters are in their heads, and when they're working with each other, they're just working with the actors, and they're not looking at the rest of it. It's all very in their minds. And a lot them actually like going back to those old workshop days and just stripping it down. So we've found a lot of acceptance from the actors that have picked it up and have been working with it.