A Supporting System of Choral Singing for Visually Impaired Persons Using Depth Image Sensor

Supporting system of choral singing Visually impaired persons may find it difficult to take part in a chorus or other singing group because they cannot see the beat indicated by conductor’s hand movements. This paper provides a chorus support system for visually impaired persons using depth image sensor. This consists of an electric music baton with an acceleration sensor, a radio module, haptic interface devices with vibration motors, a depth image sensor, and a PC. The electric music baton transmits the signal indicating the conductor’s motion to visually impaired players based on sensor acceleration. Since the conductor must give individual instruction to player, we use a depth image sensor to indicate the direction in which the conductor’s baton points. This direction is estimated based on the conductor’s posture. We also attempted to develop a chorus support system without using the electric music baton. The beat is obtained by the maximum velocity position of the conductor’s hand motion using a depth image sensor. The effectiveness of our system is clarified by several experimental results.