Joint-Temporal Heterogeneous Human Motion Decomposition

Human body motion contains various information and meanings, including individual’s emotions, habits, and health conditions. Performing a detailed analysis of human body motion may reveal rich and systematic information that benefits many computer vision tasks such as action recognition, motion style transfer, and action generation. Action segmentation considers temporal separation or a combination of body motion. In reality, actions do not always involve the movement of whole body joints. For example, punching action involves one or both arms movement regardless of any movement in the lower body parts. Frequently, more than one actions is performed simultaneously in different parts of the human body (punching while walking). Furthermore, human motions include other small movements such as habitual gestures, which are unrelated to the main action. To this end, we propose to decompose a human body motion into hierarchical subcategories in joint-temporal space. In this work, we focus on machine learning-based approach, specifically in motion segmentation and style transfer to apply our proposed motion categories. Our goal is to utilize applicable motion categories that enhance the understanding and detailed analysis of body motions. Based on the expanded understanding of human body motions, we perform three computer vision tasks that present new perspective for future research in the field. Experimental results on N-UCLA and Xia datasets demonstrate that our proposed motion categories are applicable in machine learning approaches, enables detailed understanding and analysis of human body motion.