Learning of structured trajectory models with high flexibility for computer animation
Final Report Abstract
Goal of the proposed project was the development of a method for the modeling of styleparameterized complex full-body movements based on learned elementary components that model synergies. Synergies were supposed to be defined in a statistical sense, extracting them from motion capture data by applying appropriate unsupervised learning techniques. The learned components should then be applied to generate novel movements, achieving high degrees of realism and flexibility in motion synthesis. A further goal was to exploit the developed models and parameterization for psychophysical experiments that investigate the role of learned components for the visual perception of body motion. An additional specific research focus of the second funding period was the development of new approaches for the design and the analysis of the dynamical stability of the obtained systems for motion synthesis by interaction between learned dynamic primitives. This problem was addressed using the framework of Contraction theory as method for the treatment of the stability properties of composite complex nonlinear dynamical systems. In addition to this work on trajectory generation, in the context of the Paket-Antrag, we collaborated extensively with the MPI for Biological Cybernetics on psychophysical experiments on the perception of dynamic facial expressions. This work exploited learned generative models for the synthesis of photorealistic dynamic faces, studying fundamental mechanisms in the processing of dynamic faces, such as the representation of temporal order or the relevance of high-level after-effects for dynamic stimuli. Before the start of the project dimension reduction methods had been applied successfully to trajectory data, however typically resulting in representations with a substantial number of hidden dimensions (e.g. more then 10) and limited reusability of the learned modules (such a PCA components). Furthermore, it was a wellestablished hypothesis in biology, that natural motor behavior is organized in terms of low-dimensional components (synergies, movement primitives) which potentially interact in a dynamic fashion during the realization of complex behaviors. How components can be learned from motion data that allow synthesis of highly flexible behavior from few components was largely unclear. Likewise, it was unknown how such components can be parameterized in a way that makes them reusable fort the synthesis of complex behaviors by self-organization through dynamic interaction between such primitives. The integration of movement primitives over time in terms of elements of temporal sequences of individual movements had been addressed extensively in previous work, but not the integration of modules that are spatially and temporally localized, as conceptualized in the classical concept of muscle synergies. The developed solution for the problem of motion synthesis based on learned synergies is based on the combination of three major steps: 1) development of a new method for the learning of highly compact representations of motion patterns by anechoic demixing, resulting in accurate representations of complex movements by much fewer learned components than classical techniques (PCA, ICA); 2) development of a learning-based approach that generates the learned mixture components (source signals) online from canonical dynamical systems (dynamic primitives). These systems can be designed relatively flexibly, where we chose nonlinear dynamical systems that are structurally stable, maintaining their basic dynamical properties in presence of perturbations by dynamic couplings or parameter variations. 3) The development of a novel approach for the analysis and design of the stability properties of networks of such dynamic primitives, which are capable of generating complex behaviors, exploiting the framework of Contraction Theory The accomplished work shows that the developed approach is suitable for the synthesis of complex full-body movements by relatively simple mathematical models, which (opposed to many other approaches in the literature) are accessible to a systematic treatment of dynamic stability properties. The computational feasibility of the developed approach was demonstrated by generation of complex collective behaviors of groups of articulated agents, integrating primitives for periodic and non-periodic movements. The generated behaviors include different styles of locomotion, navigation and speed-control, and the combination of locomotion with arm movements during dancing figures. In addition, the design and analysis of dynamic stability properties was illustrated for a set of scenarios with locomoting crowds that realize coordinated collective behaviors by self-organization of consensus behaviors. To our knowledge, our work represents the first control-theoretical treatment of consensus problems including agents with this level of complexity, which articulate and in addition show different variations of gait styles (curved walking, variation of step length and frequency). Our theoretical work was complemented by psychophysical experiments using the developed mathematical parameterization of complex body movements for the study of the motor patterns and the perception of emotional body movements. We found critical sets of kinematic features, and specific subsets of degrees of freedom that show characteristic changes of motion for particular emotional gait styles. Careful controls showed that the extracted feature changes are really emotion-specific and do not just represent side-effects of speed differences between the different gaits. As result of this work we could show that the visual system extracts critical features that correspond to the ones that are critical for an accurate and sparse representation of the corresponding joint angle patterns. In addition, the analysis revealed a previously unknown right-left asymmetry in the execution of emotional body movements, which could be confirmed by detailed kinematic analysis and for which we were able to show that it influences the perception of emotional body movements, and which support the right-hemispheric dominance in the control of emotional body movements. A second set of psychophysical experiments was realized in collaboration with the MPI for Biological Cybernetics in the context of collaboration. This work investigated high-level after-effects in the perception of dynamic facial expressions. Based on a generative model for photo-realistic facial expressions that was developed at the MPI, we investigated if adaptation with dynamic ‘anti-expressions’ results in after-effects facilitating the recognition of the original expressions. We found (for the first time) such after-effects for dynamic facial stimuli and could show by appropriate analysis of the associated optic flow patterns that the observed effects were not based on low-level motion after-effects.
Publications
- (2008) Lateral asymmetry of bodily emotion expression. Current Biology, 18, pp. R329-330
Roether C.L., Omlor L., Giese M.A.
- (2009) Critical features for the perception of emotion from gait. Journal of Vision, 9(6), 15, pp. 1-32. Association for Research in Vision and Ophthalmology, Rockville
Roether C.L., Omlor L., Christensen A., Giese M.A.
(See online at https://doi.org/10.1167/9.6.15) - (2009) Design of dynamical stability properties in character animation. In: The 6th Workshop on Virtual Reality Interaction and Physical Simulation, VRIPHYS 09 (Eurographics workshop), Nov 5-6, Karlsruhe, Proc., pp. 85-94
Park A., Mukovskiy A., Slotine J.J.E., Giese M.A.
- (2010) Dynamic Faces: Insights from Experiments and Computation. MIT Press, Cambridge, MA, USA
Curio C., Bülthoff H.H., Giese M.A.
- (2011) Analysis and design of the dynamical stability of collective behavior in crowds. Skala V. (ed): Proc. of the 19th Int. Conf. on Computer Graphics, Visualization and Computer Vision 2011 (WSCG '2011). J. of WSCG, Vol.19, N.1-3, pp.: 69-76
Mukovskiy A., Slotine J.J.E., Giese M.A.
- (2011) Anechoic blind source separation using Wigner marginals. Journal of Machine Learning Research, 12, pp. 1111-1148
Omlor L., Giese M.A.
