Overview of the framework

Figure 1.2 shows an overview of the model of facial behaviour we have developed. First, the face has to be tracked in the video sequence (1). The appearance model parameters have then to be deduced from the tracked face (1 $\rightarrow$ 2). The trajectory formed by those appearance parameter vectors is then broken into pathlets (2 $\rightarrow$ 3) which are grouped and modelled. The trajectory is now a sequence of pathlet models (3). The sequence of pathlet models is learnt (3 $\rightarrow$ 4) by a variable length Markov model (4).

Figure 1.2: Overview of the components of the model. P is the appearance parameter space. Arrows from left to right represent the learning phase and arrows from right to left represent the generation phase. Each face from the frames of the original video sequence corresponds to a point in the space P. The trajectory of the points in P is modelled by a sequence of pathlet models. The temporal relationship of the pathlet models is learnt by a variable length Markov model.

In order to generate new trajectories, a new sequence of pathlet models has to be sampled from the variable length Markov model (4 $\rightarrow$ 3). A new pathlet has to be sampled from each pathlet model in the sequence (3 $\rightarrow$ 2) to give a sequence of pathlets, that is a trajectory (2). Each point in that new trajectory in the appearance parameter space can then be synthesised (2 $\rightarrow$ 5) to give a video sequence of faces (5).