I have recently watched 'The Social Network'; a true story about the birth of Facebook directed by David Fincher. I really liked the way in which the story is portrayed through editing; it goes back and forth from the story as it happened to two different lawsuits - the story as it was told during the lawsuit between Facebook founder Mark Zuckerberg and the Winklevoss twins, and the story as it was told during another lawsuit between Mark Zuckerberg and the co-founder of Facebook Eduardo Saverin. However, a behind-the-scenes DVD also comes with the film and it was very interesting to see how they worked behind the scenes, how they rehearse and also the actors' point of view about the whole production. Apart from all this, the fact which I found most interesting was how they, in a certain sense, "made" the Winklevoss twins. They couldn't find the twin actors they needed so instead they hired two different actors, Armie Hammer and Josh Pence, and they replaced Josh Pence's face with Armie Hammer's face using CGI. Here below is the section of the DVD which talks about this.
http://nymag.com/daily/entertainment/2011/02/social_network_winklevii.html
I further researched this on the internet since the DVD only gives brief information to try and understand further this technique and I found the following article;
In ‘The Social Network’, Actor Armie Hammer Played 2 Twin Brothers, Dubbed By Zuckerberg As “The Winkle-vi”
Although it was Armie Hammer’s face on both twins, it was not his body. A model named Josh Pence (also extremely tall) was acting in each scene that was shot, right next to Hammer. Afterward, Hammer’s face was superimposed onto Pence’s.Word is, the two worked together for 10 months, learning to emulate each other in, especially, movement. An acting coach was used. They trained together also, so as to create as similar physiques, as possible.
- Actor Armie Hammer and Model Josh Pence
According To Director David Fincher, Using Twin Actors Was Preferred…
CGI was a last resort, but they couldn’t find two twins who were also actors, that fit the breakdown. The real Winklevoss twins are 6’5″, Olympic athletes, intelligent, and well, clearly ‘WASP’-y.
The Real Winklevoss Twins (Getty Photo)
The director insists that he didn’t use all this CGI wizardry just for kicks. “For a long time, I held out for this idea that we were going to find two 6’5″ 220-pound scullers who were going to be able to act,” he laughs. “So we looked and we looked and we looked and finally, probably about four weeks out from shooting, I just said, this is crazy. We’re never going to be able to get [it]. We need an actor. We need one person to play two people.”
Many Actors Find It Hard To Watch Their Own Performances.
How do you think Armie Hammer felt, watching two of himself, acting onscreen, at the same time, in The Social Network?
Actor Armand Hammer As A Winklevoss
The Social Network Used A Unique ‘Twinning’ Technique
…Interesting that director David Fincher winged it some, as they filmed. That’s in the article below too.‘Benjamin Button’ was also directed by David Fincher. That’s a film that also included some tricky CGI, enabling Brad Pitt to age backwards, onscreen. That film had a lot more money to spend on the special effects, besides a different CGI goal. Creating the illusion of twins, apparently, was an easier task.
Here’s how it was done, by a special effects group called Lola:
Surprisingly for anyone who knows the tech behind the Oscar winning work of Benjamin Button, Director David Fincher’s previous film, Lola ended up producing perfect twin face replacement in ‘The Social Network’ without a single HDR sample from on set. In fact, while it might appear after Benjamin Button face replacement was ‘solved’ , – the approach taken on this film was actually technically quite different.
Before we investigate why, one needs to understand the two processes. ‘Benjamin Button’ worked primarily on having an actor’s head replaced with a fully digital version of Brad Pitt’s head. A digital version based on Brad Pitt’s performance but fully rendered in 3D. HDRs were taken on set not in just one position but through a range of positions. Brad Pitt was scanned in an ICT Lightstage allowing for a near infinite range of lighting environments to be recreated and derived from the sampled Lightstage session. The HDRs from the set were then used to produce a frame of Pitt’s real head with the correct HDR lighting.
Actor Brad Pitt As Older Benjamin Button
a) a digital animated ‘aged’ Brad Pitt
b) a sampled Brad Pitt but computed or dialed in with the correct lighting – side by side.
For ‘The Social Network’, Lola’s approach was actually quite different. The problem in The Social Network was to have twins on screen played by the same actor. This is a problem that has appeared and been attempted by almost countless vfx artists since visual effects entered film making, Dead Ringers, Multiplicity, The Parent Trap, Adaption , The Man in the Iron Mask, Hot Fuzz etc, and with the exception of Matt Lucas as Tweedledum and Tweedledee in Tim Burton’s Alice in Wonderland, – these rarely involved attempting face replacement, (and even then Alice is a special case due to costume).
…The solution was not to replace Pence’s head with a digital head matched with HDRs and animation to Hammer’s performance. Instead it was to map Hammer’s face onto Pence’s face.
In the words of Lola VFX supervisor Edson Williams, David Fincher worked all this out, ” a lot of this was his idea, the basis of this technique was his, it was his fundamental idea to project on to geometry, my job was just to simplify it, he did everything he could to help us, apart from shoot HDRs ”
Steps for face replacement:
Stage 1: Pre-production The actors Pence and Hammer train to both move and look alike. Gym work and acting training produced two body shapes that matched the close similar physical presence of identical twins.
Stage 2: The two actors simply acted out the scene with Pence wearing dots on his face. “Fincher described the effect as the ‘hockey’ mask”, explained Williams. As the plan was only
replace the face, hats and head bands were used to provide a clean line for Lola.
replace the face, hats and head bands were used to provide a clean line for Lola.No one from Lola was on set for the twin’s scenes and no HDRs or special measurements were taken. (Although Fincher is extremely CGI savvy and one could argue no Lola VFX presence was needed).
Stage 3: Lola worked based on just the principal photography and any on set texture photos and production stills to recreate the lighting of the set, as it would appear on the face of Pence as he acted. “We just had the final principal photography, but nothing shot for us, we did have a lot of reference shots – just because Fincher is very thorough” points out Williams. This CGI lighting would need to animate, if say Pence walked past a door or window. At this stage a scanned version of Pence (not Hammer) is used to check accuracy. “We just recreated it roughly, we just built a simple mockup, it was just a general look of the environment”, adds Williams.
Pence’s real head is object tracked primarily in PFtrack. In a perfect world the real Pence and his digital double would match perfectly in lighting and orientation. Once this match was achieved that head is discarded – but the lighting design is used in the next stage and the object track was kept.
Stage 4: This lighting information obtained from 3D was used to program a rock concert
style DMX lighting board. Actor Armie Hammer was then filmed delivering his performance sitting in a chair filmed with 4 Red cameras. To obtain the correct eye line a back projected 30ft x 12ft screen was placed in his eye line and a projected dot was animated to move – providing the correct eye line position.This was also derived from the 3D animation done in stage 3.” We would break down the quicktime of the performance frame by frame, and control the lights, …we had a bank of 12 lights around him (Hammer) ” says Williams, but he points out that at this stage there was some flexibility. “You had to be pretty good, but if you got within 80% you were doing pretty good, you actually had a surprising amount of leeway”. That being said it is still an art to match lighting “Michael Watson helped me most with the lights, he’s worked with David Fincher in the past, he is very Red savvy a very good cameraman, he helped me the most with matching the lighting”.
Stage 5. The four Red cameras are combined to produce one full face texture mask, with the Red captured performance with the correct lighting from the DMX panel – but the orientation of Hammer’s head during the performance is unimportant as this moving textured mask is now just a disembodied 3D texture.
Stage 6: A scanned version of Hammer’s head is now carefully animated to match the 4 camera filmed performance. The digital animated head is hand animated to do this, but the animator has the action filmed from 4 angles to reference. Josh Singer was the Lead 3D artist. The digital head needs to animate to match the performance but it does not need to be lit in a CG sense as it will have the combined 4 camera RED texture map projected on it.
For example if the digital Winklevoss twin smiles – the CG face moves to match the smile – but the subtle skin texture creases and lighting all come from the live action projected performance. Interestingly while the cg nose and ears need to align to the projected texture, -
the mouth was not internally modeled, the mouth was just a flat surface that the interior mouth texture was projected on to.
the mouth was not internally modeled, the mouth was just a flat surface that the interior mouth texture was projected on to.Stage 7. The original Pence head PFtrack is now applied to the animated – texture projected digital Winklevoss face. This provides a moving – correctly positioned, correctly lit, very real face mask.
Stage 8. This face mask is now integrated with the original background plate. In some cases
this meant mesh warping the body double’s ears, “as Arnie’s ears were smaller, and that’s more of a traditional Lola kind of thing, we do that in traditional flame work”. They did not ever need to use digital hair, “we used the hair of the body double” says Williams.
this meant mesh warping the body double’s ears, “as Arnie’s ears were smaller, and that’s more of a traditional Lola kind of thing, we do that in traditional flame work”. They did not ever need to use digital hair, “we used the hair of the body double” says Williams.Then the team remove any elements such as Pence’s chin that tended to be slightly longer than Hammer’s chin. “we did do the last 15% in the comp”, he adds ” we were able to get pretty close with just the 12 lights and the DMX”.
Stage 9; This comp was then beautifully and very accurately color graded to match the on stage multiple RED performance to the exact lighting of where Pence’s body was in shot. While the technique would get the artists most of the way there, subtle differences would still need to be adjusted, shadows or highlights added (some shine or spec highlights were rendered digitally using standard CG approaches).
“It is really interesting how different their faces were tonally, as far as colour correcting, I mean two guys standing next to each other, Arnie’s face was always a bit more magenta.. it looked a bit odd when you AB’ed with the original body double but when you look at Arnie standing right next to him – it matched… it is pretty interesting how people look different in the same light”.
So why did Fincher pick this route over the Benjamin Button route?
The good point from Williams point of view is how much this approached allowed Fincher to work with Hammer to get a great performance, “David Fincher was being so specific (for example he’d say) be a bit more pensive on this take… he was getting exactly the performance he wanted, and the actor was only worrying about his performance, he wasn’t worried about where he was walking or landing on his mark, – he did not have any of those (technical) concerns, his only concern was delivering the line David was wanting him to recreate… it was a very controlled way to get an accurate performance”
It is William’s opinion that David Fincher does not like HDRs on set. It is no secret that complex HDR
Director David Fincher behind film camera
In The Social Network, by design, the actors were very similar to start with in both stature and of course importantly in age. In breaking with such a breakthrough proven method and adopting a new innovative approach Fincher simplified on set. The Social Network is believed to have been only a $50 million film vs an estimated three times that for Button…
http://www.hollywoodactorprep.com/blog/2010/11/tech-stuff-acting-as-two-different-characters-on-same-movie-screen/
