Even Driven Multimodal Augmented Reality based Command and Control Systems for Mining Industry

The mining industry is confronted with obligations to improve worker safety and provide more effective maintenance of the new-age mining machineries. It is also essential to develop reconnaissance mission to support emergency rescue operations. An approach to ensure the mine workers safety using AR (Augmented Reality) with remote monitoring CCS (Command Control System) is proposed in this paper. This method provides an interactive and augmented real time instantaneous personal safety solution. The benefit of augmented reality is blended with real time mining environmental sensor data using multimodal learning approach to predict the emergency situation instantaneously. Environmental mining sensory information is processed with Complex Event Processing (CEP) engine to derive high level events that trigger the alarm for the emergency situation. The proposed event driven multimodal AR based CCS, outperforms the existing emergency prediction solutions that use sensor fusion or deep learning AR.