Effects of facial expressions on impression evaluation for Japanese stuttering voices

Many studies have highlighted that speech disfluencies, such as stuttering, can negatively affect communication. In this study, the authors investigated listeners’ impressions of Japanese words spoken with stuttering characteristics. We investigated the manner in which different facial expressions can contribute to improving these impressions. Recordings of male and female voices of short Japanese words were modified to have stuttering characteristics by using sound editing software. Then, the listeners’ impressions of the modified voices were examined by using the semantic differential method. The impression evaluation scores of the stuttering voices were compared with those of the original voices (without stuttering). In addition, the authors conducted an experiment to investigate whether the listeners’ impressions of stuttering characteristics could be changed by facial expressions. The results of this study showed that the listeners evaluated the stuttering voices much more negatively than the ones without stuttering characteristics. However, they rated the same stuttering voices very positively when the voices were accompanied by videos of smiling facial expressions. When presented with sullen facial expressions, the listeners returned negatively rating to the stuttering voices. These results suggest that the quality of communication can be highly improved when a speaker smiles, even if the speaker’s speech is not fluent. The result can be useful not only for human–human communication but also for human–machine communication.