HMM-based Temporal Difference Learning with State Transition Updating for Tracking Human Communicational Behaviors

In our original system, we used hidden Markov models (HMMs) to model rough gesture patterns. We later utilized temporal difference (TD) learning to adjust the action model of the tracker for its behavior in the tracking task. We integrated the above two methods into an algorithm by assigning state transition probability in HMMs as a reward in TD learning. Identification of the sign gesture context through wavelet analysis autonomously provides a reward value for optimizing the attentive visual attentive tracker's AVAT's action patterns. A bound of state value functions as a constraint factor for the updating procedure in TD models has been determined to recognize whether predictive models need to be updated according with action models. Experimental results of extracting an operator's hand sign sequence during natural walking demonstrates AVAT development in the perceptual organization framework.