The VLMM tree for pathlet states: an explanation of the behaviour

One of our aims is to be able to understand the behaviour of the face in the original video sequence, by just looking at the model. The model should be able to explain the behaviour in an explicit way.

In this section, we show how the behaviour can be graphically represented. We take trajectory T2 as an example.

We have seen in section 5.5.2 that pathlet models can be represented effectively by generating a hundred pathlets from the pathlet model. We can then use this representation to draw a VLMM tree that stores the probabilities of pathlet model sequences. Such a tree is drawn on figure 7.2.

Figure 7.2: VLMM tree of pathlet states. The tree has been generated by learning the sequence of pathlet models extracted from trajectory T2. Only a part of the tree is shown.

Each node on that tree stores a probability of a pathlet state, that is the probability of a sequence of pathlet models. In a way similar to the VLMM trees of letters shown in chapter 6.5, the tree has to be read from the leaves to the root node. For instance, the node A1 in figure 7.2 stores the probability of the sequence of pathlet models referred as A, which is composed of the pathlet models A1, A2 and A3 in that order. Similarly the pathlet model B1 stores the probability of the sequence of pathlet models B, composed of the pathlet models B1, B2, B3 and B4 in that order.

With this representation of the VLMM tree, we can see what the probabilities stored in the tree represent. The global shape on the part of trajectory encoded in each node can be seen directly from the tree. We only need to concatenate generated pathlets from the pathlet models represented in nodes A1, A2 and A3 to understand the possible trajectories generated from the sequence of pathlet models A.

Figure 7.3 illustrates the way the VLMM tree has to be read for the sequence of pathlet models B. The possible trajectories described by the sequence of pathlet models B is the concatenation of generated pathlets from the pathlet models B1, B2, B3 and B4. The arrows represent how the trajectory evolves through time. First we use points generated with the pathlet model B1, then points generated with the pathlet model B2 and so forth. Since one hundred pathlets have been generated from each groups on figure 7.3, the resulting drawing represents the distribution of possible trajectories that can be generated from the sequence B (the arrows show its direction). Another example of a sequence of pathlets extracted from the VLMM tree of figure 7.2 is shown on figure 7.4.

Figure 7.3: Representation of the sequence B, highlighted in figure 7.2. The trajectory generated by this sequence is a concatenation of pathlets generated by the groups B1, B2, B3 and B4. The arrows shows how the trajectory evolves throw time following pathlets from the groups B1, B2, B3 and then B4. The original trajectory used to teach the model is drawn in grey.

Figure 7.4: Representation of the sequence A hihglighted in figure 7.2. The trajectory generated by this sequence is a concatenation of pathlets generated by the models A1, A2 and A3. The arrows shows how the trajectory evolves throw time following pathlets from the models A1, A2 and then A3. The original trajectory used to teach the model is drawn in grey.

Note that on figures 7.3 and 7.4, the directions of the pathlets generated from the pathlet models have been highlighted with arrows. The representation of the pathlet models in figure 7.2 does not show those directions. It could be added to the representation in figure 7.2 by drawing the direction of the mean pathlet of the model. The direction could also be encoded in colour.

However, in this simple example, it is easy to guess the direction of the generated pathlets. The end of a pathlet should correspond to the beginning of the next pathlet. We can deduce the directions of pathlets by matching beginnings and ends of generated pathlets.

Looking at the probabilities stored in the VLMM tree, we can easily explain the behaviour encoded in the original trajectory. This gives an advantage over other models of behaviour that usually do not provide such an explicit and structured modelling, for instance models based on the autoregressive process.