Logo
ScienceStack
Table of Contents
Fetching ...
Sign In

TMFNet: Two-Stream Multi-Channels Fusion Networks for Color Image Operation Chain Detection

Yakun Niu, Lei Tan, Lei Zhang, Xianyu Zuo

TL;DR

A novel two-stream multi-channels fusion networks for color image operation chain detection in which the spatial artifact stream and the noise residual stream are explored in a complementary manner, to effectively learn richer discriminative representations of the operation chain.

Abstract

Image operation chain detection techniques have gained increasing attention recently in the field of multimedia forensics. However, existing detection methods suffer from the generalization problem. Moreover, the channel correlation of color images that provides additional forensic evidence is often ignored. To solve these issues, in this article, we propose a novel two-stream multi-channels fusion networks for color image operation chain detection in which the spatial artifact stream and the noise residual stream are explored in a complementary manner. Specifically, we first propose a novel deep residual architecture without pooling in the spatial artifact stream for learning the global features representation of multi-channel correlation. Then, a set of filters is designed to aggregate the correlation information of multi-channels while capturing the low-level features in the noise residual stream. Subsequently, the high-level features are extracted by the deep residual model. Finally, features from the two streams are fed into a fusion module, to effectively learn richer discriminative representations of the operation chain. Extensive experiments show that the proposed method achieves state-of-the-art generalization ability while maintaining robustness to JPEG compression. The source code used in these experiments will be released at https://github.com/LeiTan-98/TMFNet.

TMFNet: Two-Stream Multi-Channels Fusion Networks for Color Image Operation Chain Detection

TL;DR

A novel two-stream multi-channels fusion networks for color image operation chain detection in which the spatial artifact stream and the noise residual stream are explored in a complementary manner, to effectively learn richer discriminative representations of the operation chain.

Abstract

Image operation chain detection techniques have gained increasing attention recently in the field of multimedia forensics. However, existing detection methods suffer from the generalization problem. Moreover, the channel correlation of color images that provides additional forensic evidence is often ignored. To solve these issues, in this article, we propose a novel two-stream multi-channels fusion networks for color image operation chain detection in which the spatial artifact stream and the noise residual stream are explored in a complementary manner. Specifically, we first propose a novel deep residual architecture without pooling in the spatial artifact stream for learning the global features representation of multi-channel correlation. Then, a set of filters is designed to aggregate the correlation information of multi-channels while capturing the low-level features in the noise residual stream. Subsequently, the high-level features are extracted by the deep residual model. Finally, features from the two streams are fed into a fusion module, to effectively learn richer discriminative representations of the operation chain. Extensive experiments show that the proposed method achieves state-of-the-art generalization ability while maintaining robustness to JPEG compression. The source code used in these experiments will be released at https://github.com/LeiTan-98/TMFNet.
Paper Structure (19 sections, 11 equations, 12 figures, 17 tables)

This paper contains 19 sections, 11 equations, 12 figures, 17 tables.

Table of Contents

  1. Introduction
  2. Preliminary
  3. Proposed Method
  4. Problem Formulation
  5. The Architecture of TMFNet
  6. Spatial Artifact Stream
  7. Noise Residual Stream
  8. Feature Fusion Module
  9. Experimental Result
  10. Experimental Setup
  11. Dataset
  12. Operation Chain Detection of Non-JPEG Images
  13. Operation Chain detection of JPEG Images
  14. Evaluation of Model Generalization Ability
  15. Evaluation of Model Transfer Learning Ability
  16. ...and 4 more sections

Figures (12)

  • Figure 1: Schematic diagram of the visual changes and pixel value changes of the image after two operations.
  • Figure 2: Overview of the network architecture diagram of the proposed TMFNet framework.
  • Figure 3: The distribution of pixel intensity in R-channel.
  • Figure 4: The proposed three RGB filters for image texture suppression.
  • Figure 5: Schematic diagram of the noise residual maps extraction with the RGB filters.
  • ...and 7 more figures