Real-Time Continuous Action Recognition Using Pose Contexts With Depth Sensors

We present an efficient approach for real-time continuous human action recognition with depth sensors. Instead of using the powerful but quite complex deep neural networks, our approach uses a lightweight discriminative frame-level descriptor, which is called the context pose (CP). The objective is to make our approach realistic in mobile depth sensor applications. CP integrates the context of the motion within a depth video. CP can increase the discriminative power of the frames and enable more frames to represent actions. CP is incorporated with a new part-based random decision forest (PRDF) method. The PRDF is designed to automatically select the optimal combination of body parts to represent and distinguish each action. We evaluate our approach on three classical single-action benchmark datasets. The experiments show that our approach has 200 frames/s and wins superior performances to the existing frame-level descriptors and classifiers in terms of accuracy.