Open AccessMulti‐layer fusion techniques using a CNN for multispectral pedestrian detection
Author(s) -
Chen Yunfan,
Xie Han,
Shin Hyunchul
Publication year - 2018
Publication title -
iet computer vision
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.38
H-Index - 37
eISSN - 1751-9640
pISSN - 1751-9632
DOI - 10.1049/iet-cvi.2018.5315
Subject(s) - pedestrian detection , multispectral image , computer science , artificial intelligence , convolutional neural network , pooling , pattern recognition (psychology) , convolution (computer science) , detector , computer vision , deep learning , fusion , visibility , feature (linguistics) , object detection , layer (electronics) , feature extraction , pedestrian , artificial neural network , geography , telecommunications , linguistics , philosophy , chemistry , archaeology , organic chemistry , meteorology
In this study, a novel multi‐layer fused convolution neural network (MLF‐CNN) is proposed for detecting pedestrians under adverse illumination conditions. Currently, most existing pedestrian detectors are very likely to be stuck under adverse illumination circumstances such as shadows, overexposure, or nighttime. To detect pedestrians under such conditions, the authors apply deep learning for effective fusion of the visible and thermal information in multispectral images. The MLF‐CNN consists of a proposal generation stage and a detection stage. In the first stage, they design an MLF region proposal network and propose to use summation fusion method for integration of the two convolutional layers. This combination can detect pedestrians in different scales, even in adverse illumination. Furthermore, instead of extracting features from a single layer, they extract features from three feature maps and match the scale using the fused ROI pooling layers. This new multiple‐layer fusion technique can significantly reduce the detection miss rate. Extensive evaluations of several challenging datasets well demonstrate that their approach achieves state‐of‐the‐art performance. For example, their method performs 28.62% better than the baseline method and 11.35% better than the well‐known faster R‐CNN halfway fusion method in detection accuracy on KAIST multispectral pedestrian dataset.