Authoring Illustrations of Human Movements by Iterative Physical Demonstration

Pei-Yu (Peggy) Chi, Daniel Vogel, Mira Dontcheva, Wilmot Li, Björn Hartmann
ACM Symposium on User Interface Software and Technology ACM Symposium on User Interface Software and Technology (2016)

Abstract

DemoDraw is a multi-modal approach to generate these illustrations as the user physically demonstrates the movements. In a Demonstration Interface, DemoDraw segments speech and 3D joint motion into a sequence of motion segments, each characterized by a key pose and salient joint trajectories. Based on this sequence, a series of illustrations is automatically generated using a stylistically rendered 3D avatar annotated with arrows to convey movements. During demonstration, the user can navigate using speech and amend or re-perform motions if needed. Once a suitable sequence of steps has been created, a Refinement Interface enables fine control of visualization parameters.