Aims and objectives

The aim of this work is to create a generic model that is able to synthesise both the appearance and behaviour of deformable objects such as human faces. The model will be an extension of the appearance model designed by Lanitis, Cootes and Taylor [57]. The appearance model is a probabilistic model used to describe an object in a still image by learning its shape and appearance from a database of hand labelled images of similar objects. It is described in more detail in chapter 3 of this report.

In the long term, we would like to apply the model to the study of interaction between people during conversations in order to synthesise one person as a response to the behaviour of a real person (see figure 1.1). For instance, we would expect a virtual person to close his mouth when the real person is speaking or the virtual person may smile when the real person is smiling.

Figure 1.1: Goal: interaction with an avatar. In an ideal system, a camera films the user and an avatar is displayed reacting to the user's expressions. Our system does not model the interaction.

For the purpose of this thesis, we bound ourselves to the generation of facial behaviour of one person. The facial behaviour is learnt from a video sequence of the person. The resulting model can be used to generate new video sequences of the same person exhibiting a similar behaviour.

We wish to avoid using frames from the original video sequence when generating behaviours, since people are good at noticing loops and frames reappearing several times. We also want the model to be explicit. We want to be able to understand the behaviour of a face in the video sequence by looking at representations of the different components of the model.