Image Manipulation Detection and Localization Based on the Dual-Domain Convolutional Neural Networks

In multimedia forensics, many efforts have been made to detect whether an image is pristine or manipulated with high enough accuracies based on specially designed features and classifiers in the past decade. However, the important task for localizing the tampering regions in a fake image still faces more challenges compared with the manipulation detection and relatively a few algorithms attempt to tackle it. With this in mind, a technique that utilizes the dual-domain-based convolutional neural networks (D-CNNs) taking different kinds of input into consideration is proposed in this paper. In the proposed framework, two sub-networks, named the spatial-domain CNN model (Sub-SCNN) and the frequency-domain-based CNN model (Sub-FCNN), are designed and trained, respectively. With the well-trained parameters, a transfer policy is applied to the training process of the D-CNN. While CNNs are capable of learning classification features directly from data, in their standard form they tend to learn features related to the image's content. To overcome this issue in image forensics tasks, a new image pre-processing layer is proposed to jointly suppress image's content and adaptively learn manipulation detection and localization features. After investigating the properties of datasets, two post-processing operations are finally proposed and compared to obtain the final results of the pixel-wise manipulation region localization. The D-CNNs is trained and validated using 75 percent of images in the CASIA v2.0 and tested using the remaining images in the CASIA v2.0, all images in Columbia Uncompressed and Carvalho datasets. The extensive experiments show that the proposed post-processing operations optimize the final tamper probability map, and our framework with the combination of Sub-SCNN and Sub-FCNN significantly outperforms the state-of-art techniques with the best F1 scores on the datasets.