Open AccessMulti‐focus image fusion with Siamese self‐attention network
Author(s) -
Guo Xiaopeng,
Meng Lingyu,
Mei Liye,
Weng Yueyun,
Tong Hengqing
Publication year - 2020
Publication title -
iet image processing
Language(s) - English
Resource type - Journals
SCImago Journal Rank - 0.401
H-Index - 45
eISSN - 1751-9667
pISSN - 1751-9659
DOI - 10.1049/iet-ipr.2019.0883
Subject(s) - computer science , convolutional neural network , artificial intelligence , convolution (computer science) , focus (optics) , feature (linguistics) , pattern recognition (psychology) , field (mathematics) , image (mathematics) , pipeline (software) , image fusion , operator (biology) , representation (politics) , feature extraction , computer vision , artificial neural network , mathematics , philosophy , linguistics , optics , physics , repressor , law , chemistry , biochemistry , transcription factor , political science , programming language , politics , pure mathematics , gene
Recently, convolutional neural networks (CNNs) have achieved impressive progress in multi‐focus image fusion (MFF). However, it always fails to capture sufficient discrimination features due to the local receptive field limitations of the convolutional operator, restricting most current CNN‐based methods’ performance. To address this issue, by leveraging self‐attention (SA) mechanism, the authors propose Siamese SA network (SSAN) for MFF. Specifically, two kinds of SA modules, position SA (PSA) and channel SA (CSA) are utilised to model the long‐range dependencies across focused and defocused regions in the multi‐focus image, alleviating the local receptive field limitations of convolution operators in CNN. To search a better feature representation of the input image for MFF, the captured features obtained by PSA and CSA are further merged through a learnable 1 × 1 convolution operator. The whole pipeline is in a Siamese network fashion to reduce the complexity. After training, the authors SSAN can accomplish well the fusion task with no post‐processing. Experiments demonstrate that their approach outperforms other current state‐of‐the‐art methods, not only in subjective visual perception but also in the quantitative assessment.