Transfer Learning and Hyperparameter Optimization for Instance Segmentation with RGB-D Images in Reflective Elevator Environments

Elevators, a vital means for urban transportation, are generally lacking proper emergency call systems besidesan emergency button. In the case of unconscious or otherwise incapacitated passengers this can lead to lethalsituations. A camera-based surveillance system with AI-based alerts utilizing an elevator state machine can helppassengers unable to initiate an emergency call. In this research work, the applicability of RGB-D images asinput for instance segmentation in the highly reflective environment of an elevator cabin is evaluated. For objectsegmentation, a Region-based Convolution Neural Network (R-CNN) deep learning model is adapted to use depthinput data besides RGB by applying transfer learning, hyperparameter optimization and re-training on a newlyprepared elevator image dataset. Evaluations prove that with the chosen strategy, the accuracy of R-CNN instancesegmentation is applicable on RGB-D data, thereby resolving lack of image quality in the noise affected andreflective elevator cabins. The mean average precision (mAP) of 0.753 is increased to 0.768 after the incorporationof additional depth data and with additional FuseNet-FPN backbone on RGB-D the mAP is further increased to0.794. With the proposed instance segmentation model, reliable elevator surveillance becomes feasible as firstprototypes and on-road tests proof.