Generating Local Temporal Poses from Gestures with Aligned Cluster Analysis for Human Action Recognition

The use of pose estimation for human action recognition has seen a resurgence in previous years, due in part to the natural representation of the activity as a sequence of key poses and gestures. The use of sequence alignment techniques has aided the process of comparing between sequences of differing temporal rates, with aligned cluster analysis segmenting an observation into lower level action primitives. We suggest that the representation of a given action class via its lower level gestures can help to identify the higher-level action class label. We therefore present a method for the generation of key poses via the initial segmentation of an action class into gestures that are similar across numerous observations. We treat all training observations as a single observation in which there are repetitions of the same action class. By applying segmentation, we then identify common gestures across the class, which are used to generate the key poses we optimize via evolutionary programming. Global recognition rates of 97.4% are achieved using a subset of the MSR Action3D dataset. We then expand the method to recognize interaction events between two individuals using the SBU Kinect Interaction dataset, achieving recognition rates of 83.9% and over 96.4% when observing the first 6 classes.