The 2D PCA initialization results in a significant self-intersection on the right, which corresponds to identical (or very close by) input values mapping to different output values in the dynamics function.  This is treated as noise in a first order dynamical model.   

GPDM with a linear+RBF kernel.  The nonlinear kernel produces a limit cycle despite large jumps in the 2D latent trajectories (Fig 3a).  The blue data points are moved by the optimization to reduce reconstruction error at the cost of a few large jumps in the latent space dynamics.  The GPDM does a reasonable job at reconstructing the walk cycle, but notice the jerkiness in the right foot of the character at ground contact. 

A GPDM based on the same data, but sampled at 120 frames per gait cycle.  Discontinuities in the latent curves cause the simulation to get stuck at a local attractor (Fig 3c).