The Case of Motion Capture Animation

So far this month we’ve talked about the classical animation of Don Bluth, and the computer-generated animation from Blue Sky Studios. But today, we’re covering one of the most interesting and (and possibly creepiest) animation types out there: Motion Capture!

Over the past several years, Motion Capture (or MoCap) and Performance Capture have been popping up in live-action and animated films. Examples include Andy Serkis’s performance as Gollum in the Lord of the Rings franchise, Andy Serkis as Ceasar in the Planet of the Apes franchise, or Andy Serkis as…oh, well, you get it. But even though Motion Capture has proven to be a valuable tool in live-action films, the technique provides an interesting aesthetic to animated films as well. 

Back in the 2000s, Motion Capture animated movies became somewhat of a fad. Several major animated productions (many of them involving Steven Spielberg or Robert Zemeckis) used the technology within 10 years. This week, we each picked an animated movie from this era that used motion capture technology to transport the physical performances of actors into completely animated productions. 

So slip on your motion capture suits and head into the studio, we’ve got some animated films to learn about!

What is Motion Capture?

Sinbad: Beyond the Veil of Mists Trailer

  • Motion capture is a method of recording an actor’s performance so that it can be transferred to a computer-generated character on screen. Instead of a character completely created by an animator, the final result is a collaboration between animators/visual effects artists, and the actor playing the character. 

  • You might be familiar with the concept of motion capture, especially as it increased in popularity over the last 20 years or so. But animated films made with motion capture are rarer than live-action films with motion capture elements, even though you could argue that motion capture as a technique is a form of animation. 

  • In the 1860s, when Edward Muybridge lined up cameras to capture the movement of a running horse, he effectively proved that creating a moving photograph was possible. This technique to record movement was one of the earliest examples of cinematography, and possibly the beginning of motion capture. 

  • Although the modern definition of MoCap involves CGI, some earlier forms of the technique began in the early 20th century. Around 1915, animator Max Fleischer invented Rotoscoping (you can learn more about Fleischer in our history of animation episodes). Rotoscoping is the process of animating over live-action reference footage to capture the exact movement of a subject (sounds a lot like MoCap, right?) 

    • Fleischer noticed that motion in animation wasn’t fluid or realistic, and to solve that problem he created a device that allowed him to animate over frames captured by a film camera.

    • Fleischer had his brother dress in a clown costume and dance on the roof of his house in front of a white sheet. The rotoscope consisted of a film projector hooked up to a car headlamp (to increase brightness). The animator would face the projector with a screen covered in tracing paper. Each image of the dancing clown appeared on the screen, ready for the animator to trace over the movement. 

    • Rotoscoping changed animation, and when the patent expired, other studios began rotoscoping, too. The most famous examples came from Walt Disney. 

  • In the 1950s, animator and pioneer in electronic animation Lee Harrison III created the first Motion Capture suit. He put potentiometers on a suit, which captured and animated motion, recording it on a CRT monitor. This looked no more advanced than a glowing stick figure. The technology continued to develop over the next few decades. By the 1980s, the process involved several cameras and markers on the actors, especially due to advancements in biomechanics at Simon Frasier University and MIT’s development of the “graphical marionette”. This involved LED lights attached to a bodysuit. An optical motion capture system rigged with two cameras then recorded an actors’ movement.  

    • The cameras were large and expensive, and the process of assigning the markers was painstakingly difficult. Because of this, the development of MoCap technology slowed. It wasn’t until the late 1990s and early 2000s that this technique would start to appear in major film productions. 

  • Whether you hate him or hate him, Jar Jar Binks is an important piece of film history, as The Phantom Menace was the first full-length film to include a completely CGI motion-capture main character. Shortly after, MoCap made its way into animated productions, including Sinbad: Beyond the Veil of Mists from 2000. 

  • But in 2004, Robert Zemeckis turned heads when he directed one of the first films ever made using entirely performance capture: The Polar Express. 

The Polar Express

  • Synopsis 

    • Santa Claus does not exist. Or does he? For one doubting boy, an astonishing event occurs late on Christmas Eve night. He lies in bed hoping to hear something like the sound of reindeer bells from Santa's sleigh. To his surprise, he instead hears a steam engine's roar and whistles just outside his window. A mysterious conductor invites him on board to take an extraordinary journey to the North Pole with many other pajama-clad children. There, he receives an extraordinary gift only those who still believe in Santa can experience. 

  • The Book

    • The Polar Express was written and illustrated by Chris Van Allsburg and published in 1985. The book is now widely considered to be a classic Christmas story. It was praised for its detailed illustrations and calm, relaxing storyline. The book is set partially in Grand Rapids, Michigan, the author's hometown, and was inspired in part by Van Allsburg's memories of visiting Christmas-filled department stores as a child. 

    • The very next year, The Polar Express was awarded the Caldecott Medal and appeared on the New York Times bestseller list. By 1989, the book had sold a million copies and made the bestseller list four years in a row. Based on a 2007 poll, the National Education Association listed the book as one of its "Teachers' Top 100 Books for Children." And it was one of the "Top 100 Picture Books" of all time in a 2012 poll by School Library Journal.

  • 2004 Movie Adaptation 

    • The Polar Express was co-written and directed by Robert Zemeckis, with Chris Van Allsburg also serving as one of the executive producers. The film stars Tom Hanks, who was also one of the film's executive producers. He plays five distinct roles, with Daryl Sabara, Nona Gaye, Jimmy Bennett, and Eddie Deezen in supporting roles.

    • Castle Rock Entertainment produced the film as their first animated movie ever. Its visual effects and performance capture were done at Sony Pictures Imageworks with a budget of about 170 million, which was a record-breaking sum for an animated feature at the time.

  • Motion Capture

    • The Polar Express was the first movie ever made entirely with performance-capture technology. This technological technique had been used before, in movies such as Lord of the Rings and “Star Wars: Episode I” but never for an entire film. 

    • Thanks to this technology, Tom Hanks was able to perform many different roles; including the Hero Boy (Chris), the Conductor, and the Father. The digital artists were able to create multiple vastly different characters in the design process but relied on Hanks to bring them to life using exaggerated facial expressions and movements. 

    • It took 50 minutes every day to strategically glue 152 markers onto Hanks' face which ensured every nuance of his expressions was recorded. Hanks said: "You forget you have them on until one falls off and someone runs across the room screaming. Suddenly, your ear is hanging on the ground." All the data from the performance was sent to a computer, where the virtual actor was given a digital wardrobe and placed in a CG set.

    • Despite these unique problems associated with mo-cap, Tom Hanks fondly remembers working on The Polar Express saying in a behind-the-scenes interview, that playing multiple characters felt liberating as an actor. As he acted in one half of the scene, he could add things that were odd, strange, or specific. Then, on the other side of it, he would remember individual moments and could comment on them.

      • Fifteen years ago, the sensors were not as strong, so the motion-captured performers had to be a bit more animated themselves. Nowadays, sensors can record the slightest movement, so these more theatrical performances are no longer necessary. 

      • However, according to Zemeckis, the hardest thing for the mo-cap actors to do in the beginning was avoid the temptation of making an action too broad or pushing it to the extreme. Zemeckis knew the performance he wanted, and his goal was to shoot them with as little interpretation by the animators as possible. One exception to this rule was body dynamics that often had to be animated (or re-animated) if there was some external force acting on a character that was not captured on set. The most common case in this film was the addition of upper body animation to create the illusion of being on a moving train, especially when the characters are on the roof. 

        • If you take a look at some behind-the-scenes footage you will get a taste of Tom Hanks’s overacting skills. The behind the scenes footage is included in the video seen above.

    • The most difficult part of animating was creating realistic-looking hair. You can’t attach motion sensors to hair, so the artists behind the film had to illustrate and create each and every strand from scratch, then animate it to move and fall realistically. This process was extremely difficult and time-consuming, but the creators behind The Polar Express refused to neglect a single detail.

  • Reception

    • The Polar Express was released in theaters in 2004 and grossed $286 million worldwide during its initial run and $314 million with subsequent re-releases. It was then later listed in the 2006 Guinness World Records as the first all-digital capture film.

    • On Rotten Tomatoes, the movie currently sits at 56% from critics and 63% from audiences. The general consensus among most critics being that though the movie is visually stunning overall, the animation for the human characters isn't lifelike enough, and the story is padded.

      • Roger Ebert gave the film his highest rating of four stars, saying, "There's a deeper, shivery tone, instead of the mindless jolliness of the usual Christmas movie", and "it has a haunting, magical quality". He also said, acknowledging comments by other reviewers, "It's a little creepy. Not creepy in an unpleasant way, but in that sneaky, teasing way that lets you know eerie things could happen."

    • The film was nominated at the 77th Academy Awards in the categories of Best Sound Editing, Sound Mixing, (before they were combined), and Best Original Song for "Believe". The song was also nominated for a Golden Globe and won a Grammy for Best Song Written for a Motion Picture, Television, or Other Visual Media. The film was also nominated at the 3rd Visual Effects Society Awards for Outstanding Performance by an Animated Character in an Animated Motion Picture.

Monster House

  • Synopsis

    • Everyone remembers that house in their childhood neighborhood, the one that could scare even the bravest of children. In Mayville, that house belongs to Mr. Nebbercracker, an angry old man ready to chase anyone off of his lawn. One evening just before Halloween, neighborhood kids DJ and Chowder accidentally kick their ball onto his lawn, upsetting the old man, resulting in a heart attack. With Nebbercracker gone, DJ, Chowder, and their new friend Jenny discover that Nebbercracker’s house might be more than a dilapidated, empty structure. 

  • Making of

    • After director Gil Kenan graduated from UCLA, he began searching for the right script for his first film. During his search, he discovered a fascinating original screenplay by Dan Harmon (creator of Community and Rick and Morty), Rob Schrab, and Pamela Pettler. The men had pitched the film to ImageMovers, the studio founded by Robert Zemeckis, Steve Starkey, and Jack Rapke in the late 90s. 

    • Although the film was originally intended to be live-action, Robert Zemeckis thought that it would be a perfect candidate for motion and performance capture. When Gil Kenan expressed interest in directing the project, he approached the story with a complete understanding of the universe, and the producers knew he was perfect for the job. 

    • Kenan had experience with animation, and even though Monster House would use MoCap, he didn’t think it made sense to design the characters in a hyper-realistic way. He explained that when he closed his eyes and imagined the main character, DJ, he didn’t see the actor that played him. He saw a stylized, animated character with a lanky body, large head, and big eyes.

      • Kenan brought in Chris Appelhans, who used Kenan’s sketches to design and illustrate the main characters. A sculptor referenced these illustrations while creating 3D models of each character. Using a laser scanner, the character designers created a digital, 3D rotation of each character. Then, texture painters smoothed out any imperfections and added the skin color, freckles, and clothing details. 

      • Kenan and his team of animators began work by storyboarding the entire movie. Then, they created an animatic, which is like a moving storyboard that just gives the animators an idea of how the movie will look. The first scene they worked on was the very first scene in the movie. It took the editors four months to cut together a mock-up of the movie even before the actors were brought in. 

      • Once the actors performed their scenes, the director and cinematographer filmed the performances and cut together a performance cut with the motion and performance capture information and the live-action footage of the actors in their MoCap suits. Then came the layout phase, which was essentially the same spirit as the animatic, but with the performances shot from similar angles as the storyboard. There were essentially four full-length versions of the movie before the final product. 

    • When the actors came in to do their takes, the makeup artists placed clear plastic masks on their faces to mark where the dots would be for performance capture. They would then put on skin-tight suits fitted with dots, and wardrobe would glue helmets onto their heads. Each actor had 80 markers on their body, and 72 markers on their face. 

    • The small performance space held 200 motion capture cameras, which picked up only the movement of the sensor dots on each actor. The cast enjoyed working in the minimal space, with very few distractions. The dots form a “marker cloud” that the visual effects artists connect. This creates a skeleton of the character before the skin and other details are added. After the stylized animation is added, the VFX artists make sure the performances of the actors aren’t lost, and they place the characters in a completely digital environment. 

    • This is where the cinematographer and director come back in and place virtual cameras in the animated environment to “film” the animation. They can choose any focal length and any position. Then, they go to the “wheel room” where they use physical wheels to control panning and tilt. For Monster House, they even used some hand-held devices to control the virtual cameras. Their goal was to make the movie look like it was made by people, not computers. They were hoping for imperfection. 

    • For the soundtrack, Gil Kenan brought in Douglas Pipes, a collaborator and friend to score his first full-length film. Pipes has gone on to score many more films like The Babysitter, Krampus, and Trick ‘r Treat. 

    • When developing Monster House, the filmmakers treated the house as its own character. They went out, looking for the scariest houses in suburban neighborhoods, and created a conglomerate of the spookiest ones they saw. The house didn’t just have to look scary, it had to sound scary too. Sound designers set up microphones in a house, and then tore it down! This helped them capture what it would sound like as the Monster House ripped itself apart. Then, they set up speakers in a barn and loudly played monstrous noises. The sounds shook the structure, distorting and bouncing around. They recorded this as well and mixed it with the actual screams of Kathleen Turner, the actor playing the house. 

  • Starring

    • At the start of production, Gil Kenan made a wish list of all the actors he wanted for the movie. In a complete stroke of beginner’s luck, he secured every single person he asked for. Among the list was beloved character actor Steve Buschemi as Mr. Nebbercracker. Legendary comedic actors Catherine O’Hara and Fred Willard played DJ’s parents. The police duo of Officer Landers and Officer Lister were played by Kevin James and Nick Cannon. Jason Lee brought Bones to life, the rotten boyfriend of Z, DJ’s babysitter played by Maggie Gyllenhaal. Jon Heder played Skull, the arcade guru that joins the kids in their fight against the monster house. Kathleen Turner, who played the infamous Jessica Rabbit, brought Constance and the monster house to life. 

    • Our three main protagonists, DJ, Chowder, and Jenny, were played by Mitchel Musso, Sam Lerner, and Spencer Locke, respectively. Mitchel Musso went on to star in Hannah Montana, Sam Lerner currently plays Geoff Schwartz on The Goldbergs, and Spencer Locke appeared in Big Time Rush. 

    • Mitchel Musso and Sam Lerner remained friends after production and on the special features, they both declared to be each other’s best friends. 

  • Reception

    • Monster House opened in July of 2006. With a budget of 75 million, it made $141,861,243 worldwide. Domestically, it didn’t make back its budget. 

    • Roger Ebert called Monster House one of the most original and exciting animated movies he had seen in a long time, giving it his highest rating of four stars. 

    • Monster House was nominated for an Oscar for Best Animated Feature but lost to Happy Feet. It was nominated for several Annie Awards, and three Saturn Awards. 

    • Overall, it received generally favorable reviews and has a 75% on Rotten Tomatoes. There have been complaints from parents that have found the movie too scary for younger children, and lacking in any positive message. 

    • Monster House was the first completely original MoCap animated film, as it was not based on any earlier material. It was scary and strange, but in the same way, that horror has always appealed to offbeat audiences. 

Test with Snowy and Peter Jackson

The Adventures of Tintin: The Secret of the Unicorn

  • Synopsis

    • Young journalist, Tintin, with his dog Snowy finds an old model ship at a flea market. Tintin notices that it is a model of the famous ship known as The Unicorn. After purchasing it, two different people approach him about buying it from him. He says it is not for sale and heads home. Unbeknownst to him, it contains a secret message that is one of three messages that lead to a treasure. In taking the model he becomes entwined in the mystery and adventure leading him to find Captain Haddock who will be the only one to uncover the truth. 

  • The Original

    • Tintin made his first appearance with Snowy in 1929 by the Belgian cartoonist Herge or known by his real name as Georges Ramis... It was first featured in Le Petit Vingtième where Herge had been commissioned to design, supervise, and illustrate. In French, TinTin is pronounced more like “TenTen.”

    • The compilations are each known as albums and there are a total of 24. 

    • Since the comics, there have been five films made, one stop motion and two in both live-action and animation. There have also been some radio dramas and two different animated shows. 

  • Who Made the Current Adaptation

    • Until around the 1980s, Tintin was pretty much only known amongst the European countries in which it was shared as a pop culture icon. At that time Indiana Jones came out and was quite a hit. Steven Spielberg who had been the director was shown French reviews of The Lost Ark and he couldn’t read most of it but he could pick out his name and a few other words. However, he kept seeing the word Tintin throughout the reviews. He asked his assistant to have them translated and that is when he saw that it referenced Tintin by Herge. He promptly got the books and began to read. 

    • Herge was a fan of Spielberg’s. They had a telephone call in 1983 and scheduled to meet up in two weeks. Unfortunately, Herge passed away before the meet-up. His wife decided to release the rights to Spielberg. A lot happened after that before the movie was finally able to be made. When it was finally made Spielberg put a nice cameo of Herge at the beginning where he paints TinTin as he looks in the comics. 

    • This 2011 adaptation truly had a talented team behind it, especially with John Williams who did the score! The writers that worked on the screenplays were Edgar Wright (Shaun of the Dead), Steven Moffat (Dr. Who), and Joe Cornish (Attack the Block). All three of these guys were familiar with TinTin and some of them were pretty big fans.

      • As I began watching the 1991 animated show it is clear that they took a lot of inspiration. These were also based on three of the original books The Crab With the Golden Claws (1941), The Secret of the Unicorn (1943), and Red Rackham's Treasure (1944). 

    • Spielberg immediately knew he wanted the man behind The Lord of the Rings to do his special effects. 

  • The Motion Capture

    • Originally Spielberg had wanted to film it live-action but it would have been greatly stylized to be similar to Herge’s original drawings. This decision was forgone because of the number of prosthetics and make-up it would have taken.

      • Snowy even during the thoughts of live-action was always going to be CGI. This is where Peter Jackson comes in. Spielberg asked him to do a screen test for the pup. Jackson, having been a Tintin fan since childhood, took the chance to join Snowy on the screen as a live-action Captain Haddock. He was even able to use the ship from King Kong on the test. 

      • When talking about his final decision to choose motion capture Spielberg said, “It was based on my respect for the art of Hergé and wanting to get as close to that art as I could... Hergé wrote about fictional people in a real world, not in a fantasy universe. It was the real universe he was working with, and he used National Geographic to research his adventure stories. It just seemed that live-action would be too stylized for an audience to relate to. You’d have to have costumes that are a little outrageous when you see actors wearing them. The costumes seem to fit better when the medium chosen is a digital one.”

    • To help them prepare for a motion capture film Spielberg and Jackson were invited by James Cameron to see the process that was used to create the hit movie Avatar. This gave Spielberg the chance to play around with the technology and see what it was like to control the camera’s view with what looked like a humongous game controller. 

      • This technology was then improved on in time for Tintin and allowed Spielberg to control the camera and direct the shots during the actors' performances. He could be right next to the actors directing them and the camera would not see him because he did not have the mo-cap suit on.

        • He was even able to watch a monitor that gave a loose creation of the animation. 

      • With all this technology all the locations had to be built digitally first so that the actors could “move” within each of the spaces. 

        • Moving within these spaces also made it easier to have incredible scene transitions. One example is when TinTin is rowing the boat the camera spans out and that boat is shown in a puddle where a character in the next scene then steps through.

  • Starring

    • This movie is jam-packed with talent, from Andy Serkis (a mo-cap pro) to Daniel Craig, Jamie Bell, Simon Pegg, Nick Frost, and even an appearance of Cary Elwes! 

  • Uncanny Valley?

    • The film was created with motion capture so that the characters could look as similar to Herge’s vision as possible but also bring it into more real space. We get to see them as 3D versions that keep a cartoonish style. It even helps when little details are shown, like birds swirling around the pickpocket’s head after he crashes into someone else. The birds are then shown to be real because a shop owner comes out with a net to collect them.

    • At first glance, this movie looks like live-action (even my brother who came into the room thought so at first.) But with this comes opinions like what Kyle Buchanan in a Vulture article said, “Aside from the swoop in the front of his hair that lends him some cartoonish verve, Tintin looks simultaneously too-human and not human at all, his face weirdly fetal, his eyes glassy and vacant instead of bursting with animated life.”

    • Yet I think that the film does not go too far into the Uncanny Valley. Very few times was I taken out of the film by how a character looked. Roger Ebert even said, “Tintin looked human, if extremely streamlined. His face, as described by an eyewitness to a police artist, would produce a sketch of … Tintin. The other characters are permitted more detail; Thomson and Thompson, in particular, are given noses that would make W.C. Fields weep with envy.”

    • What other film could bring Simon Pegg and Nick Frost together as practically identical-looking characters, Inspectors Thomson and Thompson? That in itself is a joy.

    • It has to be mentioned that although Snowy has talking bubbles in the comics, he luckily does not talk in the film.

  • Reception

    • There are a lot of differing opinions on this movie. There seem to be several die-hard European fans of TinTin that believe this movie does not pay proper tribute to the legacy that Herge built and question the “Hollywood” look of motion capture. Others that grew up with the comics and the movies and tv shows thought that it was a fun film that brought out all the best parts of the stories. Still, others think that it falls into the uncanny valley. In theaters though it did well, with an estimated budget of 135 million dollars, grossing almost 374 million dollars worldwide.

    • The movie had a lot of people working on it that had new and old love for the original content. I think that no matter your thoughts on the movie it brought knowledge of TinTin to more people to enjoy. 



Motion Capture is still a heavily used method in Hollywood, even though the practice of MoCap in full-length animation has fallen to the wayside. Films like The Polar Express, Monster House, and The Adventures of TinTin presented interesting and exciting stories in a unique style that captured the imagination of audiences (or maybe creeped them out a little). 

Although 3D computer animation seems to be the most favored style today, movies that implement different techniques tend to stand out. Who knows, as technology improves, maybe motion capture will MOVE back into theaters again soon. 

Sources:

2000's, Animation, Film History, PodcastAdam OsburnMotion Capture, Steven Spielberg, Andy Serkis, Robert Zemeckis, MoCap, Monster House, The Polar Express, Tintin, Gil Kenan, Peter Jackson