Facial Motion Capture

More for nostalgia than any other reason, we have included these pictures to illustrate just how far we have come from the early days for "mechanical capture". Above is one of the first motion capture rigs we built for Jim Henson's Creature Shop.

A combination of rotary potentiometers, LVDTs (Linear Variable Diferential Transformer), and Infra-Red transmitter/receiver modules enabled facial expressions to be converted into electronic signals for driving animatronic robots in real time.

Part of the main design concern was ambient background light levels that can cause interference to Infra-Red systems. This has the effect of distorting the animatronic control signals as the sun came out from behind a cloud - basically messing up the characters expressions. In this version we cancelled out any background light, and successfully used it under varying sun and strobe lighting conditions.

Facial capture for Lost In Space feature film

A few years down the line we got our hands on one of the first Vicon 370 optical mocap rigs, and started to capture facial expressions using reflective markers stuck onto the actors face. Whist in the early day this was not real time, it did give us more detailed information than any electro-mechanical system. As computer processing power grew during the 90s, and mocap rigs improved, we were able to increase the resolution and accuracy of the system, and are now able to drive both mechanical and computer graphic creature faces in real time to sub-millimetre precision.

For all of the advantages of highly accurate real time facial capture with all of its lip-synch capabilities, there is one point that should not be forgotten. For many dynamic creature expressions it is often preferable to use hand control inputs as puppetry devices or key frame techniques rather than recording directly off a performers face.

Hand controls can give the puppeteer more control and enable exaggerated expressions that are ideal of an non human creature. Examples of mixing full-body optical mocap and hand controls for facial animation can be seen in movies about HDPS(TM) (Henson Digital Performance Studio).

Facial capture and hand controls are mainy used for live performances and some video game productions, however for high quality feature film work, the majority of characters facial movements are still key framed for finer control. Often human movement is not aesthetically desired, particularly for non human characters, and a more stylised performance is best generated by other techniques.

In conclusion, choosing the right technology for the right situation is critical for controlling digital and animatronic creatures in the sytle that you desire.