Open AccessCREATING AN INTERACTIVE MUSICAL EXPERIENCE FOR A CONCERT HALL
Author(s) -
Svetlana Antoshchuk,
Mykyta Kovalenko,
Jürgen Sieck
Publication year - 2018
Publication title -
computing
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.184
H-Index - 11
eISSN - 2312-5381
pISSN - 1727-6209
DOI - 10.47839/ijc.17.3.1034
Subject(s) - gesture , computer science , musical , set (abstract data type) , human–computer interaction , musical instrument , magic (telescope) , gesture recognition , augmented reality , virtual reality , representation (politics) , computer vision , artificial intelligence , multimedia , speech recognition , acoustics , visual arts , art , physics , quantum mechanics , politics , political science , law , programming language
In this paper we introduce a Kinect based posture recognition approach that can classify the user’s pose and gesture and match them to a set of predefined musical instruments. The efficiency of the approach is then demonstrated using two applications. The Virtual Orchestra system uses pose- and gesture-recognition along with Augmented Reality technology to add a virtual musical instrument into the scene, both visually and audibly: the visual representation of the instrument is placed into the user’s hands and the sound of the corresponding instrument is played. An additional functionality is that the user can control the intensity and the pitch of the sound by changing the speed of his hand or finger movements. The Magic Mirror game is the second application developed for the Berlin Concert Hall that uses the posture recognition approach to introduce the visitors to some classical music pieces and familiarize them with various classical musical instruments.