According that we have properly modeled the sub-trajectories, we will need a higher level learning algorithm that is able to reconstruct the trajectories in a probabilistic manner, by selecting the sub-trajectories models. The variable length Markov model will be tested on real data at this level. Other architecture of Markov model can also be tested. Algorithm like ID3 may be useful too.
At this stage, we will also have to find how we can chain up the sub-trajectories so that it appears like one big trajectory. The resulting trajectory has to be smooth, so we cannot just generate on sub-trajectory after the other, we also need to link them properly.
We can then apply the result of this step to improve the tracker by including search in an area where the face should be if its behaviour is typical.