Multi-label classification for image tamper detection based on Swin-T segmentation network in the spatial domain

CNN and variants

CNNs are among the most iconic neural network types, extensively utilized in various image processing tasks. They generally feature a hierarchical architecture comprising convolutional layers, batch normalization layers, fully connected layers, and additional layers. In the literature, there are many advanced and efficient neural network structures that have been introduced, e.g., AlexNet (Krizhevsky, Sutskever & Hinton, 2012), fully connected network (FCN) (Long, Shelhamer & Darrell, 2015), ResNet (He et al., 2015), HR-Net (Sun et al., 2019), and UperNet (Xiao et al., 2018). Bondi et al. (2017) trained a CNN to extract features specific to the device that captured the image. The algorithm determines that an image has been tampered if an image includes two or more features of the capturing device. Qian et al. (2021) introduced a CNN model for image tampering detection that leverages an image preprocessing algorithm. They combined the traditional support vector machine and machine learning methods to extract the features from qualification certificates for image tampering detection. To further enhance the detection accuracy of the convolutional network, the researchers adjusted the network structure by deepening the layers of the convolutional network, based on residuals, and further adding attention mechanisms. In this article, we utilize FCN (Long, Shelhamer & Darrell, 2015) to process input images of any size and eventually restore them to their original dimensions through upsampling, to further assist the network in achieving pixel-level tampering classification training. Furthermore, we improve pyramid structure of the UperNet (Xiao et al., 2018) network to fully integrate image features at different resolutions (hereafter referred to as MRF-UperNet).

Image manipulation detection/localization

Initial studies mainly focused on detecting particular types of manipulations, such as splicing (Cozzolino, Poggi & Verdoliva, 2015b; Huh et al., 2018; Kniaz, Knyaz & Remondino, 2019), copy-move (Cozzolino, Poggi & Verdoliva, 2015a; Rao & Ni, 2016), and removal (Zhu et al., 2018). Nevertheless, the precise nature of the manipulation is frequently uncertain in the practical situations. This uncertainty has driven a growing body of research focused on developing methods for general manipulation detection. Alshanbari (2021) employed Discrete Wavelet Transform (DWT) decomposition for medical images. González Fernández et al. (2018) initially calculated the likelihood of each pixel being interpolated and then applied the Discrete Cosine Transform (DCT) on small blocks of the probability map to identify tampered regions within the image. PSCCNet (Liu et al., 2021a) extracts hierarchical features via a top-down approach and assesses whether an input image has been manipulated using a bottom-up approach. RRU-Net (Bi et al., 2019) integrates residual blocks within both the encoder and decoder to address the vanishing gradient issue in deep networks and employs recurrent units in the skip connections. MVSS-Net (Dong et al., 2021) uses multi-perspective feature learning to collect information from various perspectives and applies multi-scale supervision to enhance detection accuracy by combining features across multiple resolutions. Zhou et al. (2023) employ a contrastive learning approach to tackle the issue of data insufficiency in image tampering detection. Ma et al. (2023) enhanced the method at the patch level instead of the pixel or image level, utilizing Vision Transformer for benchmarking and proposing several evaluation metrics. Qu et al. (2024) classified tampered images into two categories based on foreground and background, facilitating the automatic and precise pixel-level labeling of numerous manually forged images obtained from the web. In this article, we employ a multi-class network structure for tampering detection, achieving precise localization by capturing spatial information.

Transformer-based backbones

The Transformer architecture was first introduced in 2017. Its ability to effectively capture long-range feature dependencies quickly made it a dominant machine learning framework for natural language processing tasks (Vaswani et al., 2017). However, its application in the field of image processing is limited due to its huge computational complexity and inability to handle image information with high resolution. To decrease computational demands and fully leverage the benefits of the Transformer, some researchers innovatively introduced the Vision Transformer (ViT) (Dosovitskiy et al., 2020). ViT and its follow-ups (Touvron et al., 2021; Yuan et al., 2021; Chu et al., 2021; Han et al., 2021; Wang et al., 2021) adapted the Transformer architecture for computer vision, where its attention mechanism effectively captured long-range dependencies among different elements in a sequence, enhancing sequence processing.

In the context of image tampering detection, several recent Transformer-based approaches have shown promising results but face significant limitations. TransForensics (Hao et al., 2021) combines self-attention encoders with dense correction modules, achieving rich point interactions and multi-scale correction. However, it suffers from resolution constraints and high memory usage, limiting its practical application. ObjectFormer (Wang et al., 2022) introduces object-level attention and RGB-frequency fusion for global semantic understanding, but its limited frequency learning capability and sensitivity to compression artifacts restrict its performance in real-world scenarios. IML-ViT (Ma et al., 2023) employs a pure Vision Transformer backbone with edge supervision, offering high-resolution processing capabilities but at the cost of excessive computational requirements.

After comprehensive comparison of these approaches, we identified Swin Transformer (Swin-T) (Liu et al., 2021b) as the most promising foundation for our work. As a subsequent innovation following ViT, Swin-T introduces a sliding window mechanism that effectively balances global and local information processing while maintaining computational efficiency. Unlike TransForensics’s memory constraints, ObjectFormer’s limited frequency learning, and IML-ViT’s computational overhead, Swin-T provides an optimal balance of feature extraction capability and practical efficiency. Building upon this foundation, we enhance Swin-T with our spatial domain sensing module and multi-resolution fusion structure, specifically designed for tampering detection while preserving its computational advantages. In this article, we employ a segmentation backbone network built on Swin-T.

Network design

Spatial domain sensing module

The spatial image information is rich and diverse, not only contains content information, but also may be mixed with noise, edge details and potential tampering information. For those subtle tampering of document images that are difficult to recognize with the naked eye. Accurate detection based solely on the image’s visual information is challenging, necessitating the detection and analysis of tampering information concealed within the image. To allow the network to grasp more spatial information, the spatial domain sensing module has been designed. This module applies three different convolutional operations: SRMConv2D (Zhou et al., 2018), BayarConv2D (Bayar & Stamm, 2018) and classical Conv2D layers to the spatial information, and combine the extracted features by a simple concatenation operation.

SRMConv2D uses three SRM filters, whose weights are shown in Fig. 3. We froze the weights so that local noise features are extracted more efficiently by utilizing the difference between a pixel’s actual value and its estimated value, obtained through interpolation from adjacent pixel values, thus detecting tampering artifacts that are difficult to detect in the RGB channels.

BayarConv2D achieves effective perception of image edge information and detection of tampering traces by introducing specific constraints in the convolution operation. These constraints are shown in Eq. (1), where $w_k$ denotes the $kth$ convolution filter and the center value of the convolution filter is denoted by $(0,0)$. The constraints dictate that the center value of the convolutional filter must be $-1$, while the sum of the weights at all non-center positions must equal one. This design allows the convolutional layer to build up information with residuals to better capture edge details and potential tampering traces in the image.

(1) $$\{\begin{array}{c}{w}_k(0,0)=-1,\\ {\sum }_{m,n\ne 0}{w}_k(m,n)=1,\end{array}$$

Backbone network Swin-T

The backbone network is employed to extract high-level features indicative of tampering. This subsection refers to different models in selecting the backbone network, and comparatively analyzes the efficacey of different backbone networks such as ResNet (He et al., 2015), U-Net (Ronneberger, Fischer & Brox, 2015), HRNet (Sun et al., 2019) and Swin-T (Liu et al., 2021b), and finally selects Swin-T based on the experimental effects and training speed. Then, we introduce the relevant principle of Swin-T network and its construction process.

Principle

Swin-T (Liu et al., 2021b) is an innovative visual Transformer, functioning as a convolutional neural network built upon the Transformer architecture. Compared to the traditional Transformer, Swin-T aims to reduce the computational burden on high-resolution image processing. It employs a staged image processing strategy, where a large-size input image is decomposed into multiple small-size image blocks, each of which is called a patch. Inside the patch, Swin-T performs self-attentive computation and establishes patch-to-patch dependencies with sliding window operations. Therefore, it significantly reduces the computational complexity.

Swin-T innovatively proposes two main modules:

(1) Module for changing sequence length

Swin-T can modify the module structure, increase the number of channels, and create a hierarchical design while reducing computational load through three operations: the patch partition layer, linear embedding layer, and patch merging layer. The patch partition layer segments and stacks the channel modules, the linear embedding layer alters the feature dimensions, and the patch merging layer functions similarly to a pooling layer. The patch merging layer executes a down-sampling operation prior to each processing step, by selecting elements in the direction of rows and columns through intervals and then recombining them into new patches, and hence feature resolution reduction can be achieved. Its structure is shown in Fig. 4. Unlike the traditional pooling operation, the Patch merging layer focuses more on feature selection and reorganization. So, it can extract and deliver key information more effectively.

(2) Multi-head attention module based on windows

It primarily comprises two types of modules: Window Multi-Head Self-Attention (W-MSA) and Shifted Window Multi-Head Self-Attention (SW-MSA). W-MSA begins by partitioning the feature map into non-overlapping windows and independently executing self-attention computations within each window, thereby significantly reducing computational complexity. The W-MSA module conducts self-attention computations solely within its respective window, which may result in an inability to exchange information between windows. Therefore, the SW-MSA module is created. As shown in Fig. 5, the SW-MSA changes the relative positions of the image blocks, which is equivalent to re-segmenting the image blocks, and then calculates the self-attention between the windows. W-MSA and SW-MSA are generally used consecutively as a pair, if W-MSA is applied in the $l$-th layer, then SW-MSA is applied in the $(l+1)$-th layer. Due to the window shift, in the $(l+1)$-th layer, when determining self-attention for the windows, information exchange occurs with the windows from the $l$-th layer. This addresses the issue of inability to exchange information between different windows.

Network construction process

The Swin-T network is used to extract image tampering features, and its structure is illustrated in Fig. 6. Initially, in this article, the extracted features from spatial domain sensing module are input into the Swin-T network. These features are then divided into N non-overlapping patch tokens and concatenated along the channels to create a high-dimensional patch sequence. A Linear Embedding layer subsequently transforms the feature dimensions from $48$ to a chosen dimension C. Subsequently, these processed features are fed into a self-attentive module (i.e., Swin-T Block) with multiple sliding windows. After that, the height (H) and width (W) of each group with $2\times 2$ adjacent patches are reduced by half, i.e., transforming from H/4 × W/4 to H/8 × W/8. Meanwhile, the dimensionality of the patch tokens is doubled, i.e., increasing from $2C$ to $4C$.

In each processing stage, the features pass through either a linear embedding layer or a patch merging layer and then through several pairs of attention modules with sliding windows, and thus constructing a hierarchical tampering feature representation. Such a design reduces the computational cost while reasonably utilizing self-attention to capture the correlation between different patches and better capture the tampering traces of the document images.

Decoder network MRF-UperNet

As the backbone network deepens, the size of the image features diminishes, and the deep feature maps are rich in information but have low resolution, that make it more difficult to capture precise locations. In contrast, shallow feature maps contain clear positional information but have weak representational capacity, that make them unable to capture tampering information. To accurately detect the target location, fusing the semantic information from both deep and shallow layers is essential. Given that the proposed model adopts the structure of Swin-T, which possesses multilevel feature maps, this section introduces the pyramid structure of UPerNet (Xiao et al., 2018) as the architecture for the decoding network and incorporates the design of multiresolution fusion on the basis of this structure.

Essentially, multi-label classification for document image tamper detection is a semantic segmentation task, enabling the identification and localization of different tampered regions at the pixel level. In this article, the part of UPerNet network for region segmentation is selected and an improved UPerNet structure is designed. The primary structure of the original UPerNet is illustrated in Fig. 7A. UPerNet utilizes the standard Feature Pyramid Network (FPN) design, incorporating a top-down feature fusion pathway. FPN progressively passes and integrates high-level, semantically rich feature maps into lower-level, high-resolution feature maps, achieving multi-scale feature integration. It receives inputs from different scales extracted from the Swin-T backbone network and unifies the number of channels in the different layers by a $1\times 1$ convolution. Beginning with the top layer of the backbone network (stage 4), the most profound features are extracted and processed through the Pyramid Pooling Module (PPM) to achieve multi-scale feature fusion $F_1$, and then upsampling it to the same size as that of stage 3 by $1\times 1$ convolution, then perform the addition operation for feature fusion to obtain $F_2$. The similar process are performed on $F_2$ and $F_3$ to get the corresponding fused features. After that, the fused features $F_1$, $F_2$, $F_3$, and $F_4$ are combined using a $3\times 3$ convolution and up-sampling with a uniform scale. The features are further fused to obtain information from different levels. PPM extracts global contextual information from feature maps through multi-scale pooling operations. This global information helps the network understand the overall structure of the scene, enhancing its perception of large-scale regions. PPM emphasizes global information, while FPN focuses on multi-scale, detailed, and local features. By concatenating the outputs of PPM and FPN and feeding them into the prediction module, the network can simultaneously utilize both global and local information, enhancing the accuracy and robustness of segmentation results. In this article, we enhance the primary structure of UPerNet by incorporating new convolution and fusion modules, as illustrated in Fig. 7B. $F_1$ is fused with the module extracted in stage 3 using $3\times 3$ convolution and upsampling. This process is repeated on the subsequent modules to achieve continuous feature integration. Meanwhile, features of the same size in each stage are repeatedly utilized and continuously fused in each iteration through a $1\times 1$ convolutional join. Therefore, we can obtain fully integrating information from multiple resolutions.

Additionally, we introduce a multi-resolution fusion network (MRF-UperNet) built upon the enhanced UPerNet, drawing inspiration from the human eye’s repetitive observation process. Our goal is to detect traces of image tampering by continuously fusing high-resolution and low-resolution information. This design capitalizes on the richness of the network encoder’s features at different resolutions, and also enhances the network’s ability to perceive subtle tampering cues in images.

Dataset construction and experimental setup

The scarcity of comprehensive document tampering datasets presents a significant challenge in current research. Most existing datasets are limited in both sample size and tampering diversity. For instance, the Tampered-SROIE dataset contains only 626 samples, which is insufficient for training deep learning models effectively. Moreover, existing datasets typically focus on single tampering methods without detailed type annotations, making multi-class tampering detection particularly challenging.

To address these limitations, we developed a new dataset by utilizing two source datasets: the StaVer Dataset and SCUT-EnsExam. The former comprises document images containing various elements such as colored logos and stamps, while the latter consists of handwritten exam sheets. We implemented a systematic processing pipeline to create our comprehensive tampering dataset:

First, we standardized all images to 512 × 512 pixels through random cropping. For the StaVer Dataset, we extracted three distinct regions from each image, while for SCUT-EnsExam, we selected ten regions per image. This initial processing resulted in a base collection of images suitable for tampering manipulation.

Our tampering generation process focused on five distinct manipulation types that commonly occur in real-world document forgeries:

The final dataset comprises over 30,000 images, with approximately 80% allocated for training and 20% for testing. Each tampering category is well-represented, with distribution as shown in Table 2. Figure 1 illustrates examples of each tampering type.

Our dataset construction process followed systematic criteria to ensure diversity and minimize potential biases. We carefully selected source documents from both StaVer Dataset (containing colored logos and stamps) and SCUT-EnsExam (handwritten exam sheets), ensuring coverage of different document styles and content types. To maintain data quality and reduce potential biases, we implemented balanced sampling across tampering types (as shown in Table 2), with each type having approximately 6,000 samples except for erasing operations. The tampering operations were deliberately applied at varying locations and scales to ensure comprehensive coverage of different document regions and forgery patterns.

For comprehensive evaluation, we also incorporated the DocTamper dataset (Qu et al., 2023) as an external validation set. This dataset contains 170,000 document images in both Chinese and English, providing an independent benchmark for assessing our model’s generalization capabilities. While DocTamper lacks detailed tampering type annotations, its diverse collection of document styles and languages makes it particularly valuable for validating our model’s performance beyond the training distribution. The subsequent comparative experiments will be conducted on both our self-constructed dataset (hereafter referred to as MixTamper, as detailed in Table 3) and the publicly available DocTamper dataset to demonstrate the model’s robust performance across different document scenarios.

Our dataset offers several key advantages: it provides a substantial number of samples, includes multiple tampering types with precise annotations, and represents both English and Chinese document scenarios. The tampering operations are designed to be subtle and localized, closely mimicking real-world forgery patterns. However, we acknowledge certain limitations: the document types are primarily confined to specific categories, and the fixed image dimensions may not fully represent all real-world scenarios.

For the experimental implementation, we employed the Adam optimizer with $\beta$ values of $(0.9,0.999)$ and a weight decay rate of 0.01. The learning rate was managed using a poly strategy, initialized at 0.0001, with a linear warm-up period of 1,500 iterations. The training process incorporated both multicategorical cross-entropy and Lovász-Extension loss, weighted equally, over 300 epochs.

Visualization results

In order to show the prediction results more intuitively, tests are conducted on the MixTamper dataset and the corresponding prediction masks are generated. Some of the prediction results are selected as shown in Fig. 8, selected results are displayed from left to right: the tampered image (forgery), the ground-truth image (GT), and the prediction output of the proposed method (mask). In GT and mask, various colors are employed to distinguish between types of tampering: red for copy-move tampering (our work for copy-move forgery detection is only focus on the detection of target regions), green for splicing tampering, blue for text-generating tampering, yellow for smearing tampering, and purple for erasing tampering.

Figure 8 displays the experimental results. The first image contains a combination of five forgery types, which are challenging for existing state-of-the-art (SOTA) methods to detect. However, our proposed method can accurately identify and label each type, which demonstrates the model’s excellent performance in dealing with multiple types of tampering, especially the erasure tampering, which is detected accurately by the algorithm even though the tampered area is very small. The second image contains two types of tampering: copy-move tampering and erasure tampering. The copy-move tampering moves the text below the basketball image to the right side of the number 2, which is relatively hidden, our algorithm is still able to accurately locate the tampering position. The third image contains two types of tampering: splicing tampering and text-generating tampering. Splicing tampering is more hidden, although its detection results are not clear at the tampering boundary, it is still able to accurately identify the general location, while text-generating tampering also achieves the desired effect. The fourth image involves copy-move, splicing and text generation tampering, all of which can be accurately detected. The fifth image incorporates four types of tampering, including copy-move, splicing, text generation and erasure tampering, in which the erasure tampering and text generation tampering means are intertwined, our method not only accurately recognizes each type of tampering, but also accurately locates the tampered area. Overall, our proposed model is capable of accurately identifying tampering types even in complex tampering scenarios.

Ablation experiments and analysis

To evaluate the effectiveness of the proposed algorithm for detecting multiple types of forgery based on the Swin-T segmentation network in the spatial domain, an ablation experiments are designed. On one hand, the performance of various backbone and decoder networks is compared, and detailed adjustments to the network structure are made. This allows for an analysis of how specific network structures impact the algorithm’s performance. On the other hand, to verify the effectiveness of network optimization strategies, adjustments are made not only to the loss function but also by introducing hard sample mining and self-supervised augmentation methods. Corresponding ablation experiments are designed to support these analyses. Through these experiments, the positive effects of these optimization strategies in enhancing the performance of tamper detection algorithms are successfully verified.

Analysis of combined loss function

To understand the effectiveness of our combined loss function, we conducted detailed experiments analyzing both individual components and their combination. The cross-entropy loss $L_{CE}$ focuses on pixel-wise classification accuracy, while the Lovász-Softmax loss $L_{LS}$ directly optimizes the IoU metric. We hypothesized that combining these losses would leverage their complementary strengths.

Our experiments reveal distinct characteristics and limitations of each loss function. As shown in Table 8, the cross-entropy loss ( $L_{CE}$) shows rapid initial convergence but becomes unstable after epoch 50, leading to NaN values. This instability arises because $L_{CE}$ struggles with the inherent class imbalance in tampering detection, where tampered regions often constitute a small percentage of the image.

The Lovász-Softmax loss ( $L_{LS}$) shows slower but stable convergence throughout training. However, its final performance varies significantly across tampering types. While it excels at detecting tampering types with clear boundaries and uniform regions (Text: 83.32% IoU, Erasure: 91.98% IoU), it significantly underperforms on complex tampering types that require fine-grained feature discrimination (copy-move: 21.90% IoU, splice: 28.82% IoU), as displayed in Table 9.

Our combined loss ( $L_{CE}+L_{LS}$ with 1:1 ratio) achieves both stable convergence and superior performance across all tampering types. As evidenced by Table 8, it maintains steady improvement throughout training while avoiding the instability of $L_{CE}$ alone. This improvement can be attributed to the complementary nature of the two losses: $L_{CE}$ provides strong pixel-level supervision that helps with fine detail preservation, while $L_{LS}$ optimizes the IoU metric directly, ensuring robust segmentation of tampering regions regardless of their size. The combination particularly benefits complex tampering types, showing dramatic improvements in copy-move (72.92% vs 52.45% IoU) and splice (79.09% vs 61.00% IoU) detection compared to using either loss alone.

Comparative experiments and analysis

To highlight the advantages of the proposed method, this section offers a detailed comparison of its performance with that of the existing RRU-Net (Bi et al., 2019), DenseFCN (Zhuang et al., 2021) and PSCC-Net (Liu et al., 2021a) for image tampering detection. To ensure comprehensive comparisons, we conducted comprehensive experiments on both our MixTamper dataset and the external DocTamper dataset.

For evaluation metrics, we use IoU and F1-score. For the DocTamper dataset, which only provides binary annotations, we compute these metrics directly on the binary prediction masks. For the MixTamper dataset with multi-class annotations, we calculate mean IoU (mIoU) and F1-score as follows:

The mIoU is computed by first calculating per-class IoU values and then averaging across all classes:

(8) $$mIoU=\frac{1}{N}{\sum }_{i=1}^N\frac{T{P}_i}{T{P}_i+F{P}_i+F{N}_i}$$where N is the number of classes (N = 6 in our case, including the background class), $T{P}_i$, $F{P}_i$, and $F{N}_i$ represent true positives, false positives, and false negatives for class i, respectively.

The F1-score is similarly computed per-class and then averaged:

(9) $$F1=\frac{1}{N}{\sum }_{i=1}^N\frac{2T{P}_i}{2T{P}_i+F{P}_i+F{N}_i}.$$

We use unweighted averaging across classes to ensure that each tampering type contributes equally to the final score, regardless of its frequency in the dataset.

As shown in Table 10, our method demonstrates consistent superior performance across both datasets. On the external DocTamper dataset, which contains 170,000 diverse document images in both Chinese and English, our method achieves an IoU of 84.16% and F1-score of 91.40%, significantly outperforming existing approaches. This strong performance on an independent external dataset confirms that our model’s improvements are not overly specialized to our training set. Similarly, on the MixTamper dataset, we achieve an mIoU of 82.97% and F1-score of 90.46%, demonstrating excellent detection accuracy across different tampering types. The consistent performance between DocTamper and MixTamper (variance < 2% in both metrics) validates our model’s robust generalization capability across different document styles and languages, while the significant margin over SOTA methods (improving IoU by >11% on both datasets) demonstrates the effectiveness of our approach.

Analysis of mixed-type tampering strategy

Most existing deep learning approaches for document tampering detection focus on binary localization without distinguishing between different tampering types. For instance, ObjectFormer, a state-of-the-art method, outputs a binary mask (H $\times$ W $\times$ 1) to indicate tampered regions and uses a classifier to determine authenticity. While effective for single-type tampering detection, this approach cannot effectively handle cases where multiple tampering types coexist. In contrast, our method treats tampering detection as a unified multi-class segmentation task, where each pixel is directly classified into one of six categories (original:0, copy-move:1, splicing:2, text:3, smearing:4, erasing:5).

To validate the effectiveness of our approach and justify the necessity of mixed-type tampering training, we conducted comprehensive experiments comparing ObjectFormer and our method under both single-type and multi-type scenarios. For fair comparison, we trained ObjectFormer on our dataset following its original configuration, while only considering localization accuracy (IoU and F1 score) in multi-type scenarios since ObjectFormer cannot predict specific tampering types. Table 11 presents the performance comparison:

The experimental results provide strong evidence supporting our mixed-type tampering strategy. In single-type scenarios (specifically copy-move tampering), ObjectFormer achieves slightly better performance, demonstrating its effectiveness in traditional tampering detection tasks. However, when handling multi-type tampering cases, ObjectFormer’s performance drops significantly (IoU decreases by 15.8%), while our method maintains robust performance (only 1.4% decrease in IoU).

This performance gap can be attributed to two key factors: (1) Our unified multi-class segmentation approach preserves the spatial and semantic relationships between different tampering types (2) Training with mixed-type samples enables the model to learn complex feature interactions that commonly occur in real-world document forgeries.

These results clearly demonstrate that our mixed-type tampering strategy is not only theoretically sound but also practically beneficial. While existing methods like ObjectFormer excel in single-type scenarios, they struggle with real-world cases where multiple tampering types coexist. Our approach successfully addresses this limitation while maintaining comparable performance in traditional single-type detection tasks.

Robustness analysis

To evaluate our model’s robustness against real-world image distortions, we conducted comprehensive tests with various types of degradations:

Our analysis reveals that performance degradation varies significantly across different tampering types and distortions in Table 12. For JPEG compression, text tampering shows the strongest robustness, with only a 6.33% IoU decrease at q = 75, likely due to the high-contrast nature of text modifications. In contrast, Erasure tampering exhibits the most severe degradation (IoU drops by 43.38% at q = 75), as removed areas appear as uniform background regions that compression algorithms tend to oversimplify. Copy-move, splice, and smearing tampering show moderate but significant degradation (IoU drops of 21.13%, 23.37%, and 36.77%, respectively at q = 75), primarily due to compression artifacts masking fine-grained tampering traces.

Similar patterns emerge under Gaussian noise, where text tampering maintains relatively stable performance (7.54% IoU drop at $\sigma =0.03$) due to its distinct contrast patterns. However, copy-move and splice detection suffer more severe degradation (20.16% and 19.18% IoU drops respectively at $\sigma =0.03$) as noise obscures the subtle similarities between copied regions and spliced content. For Gaussian blur, the performance impact is generally less severe than compression or noise, but follows similar patterns across tampering types. Text tampering remains the most robust (7.09% IoU drop at 7 $\times$ 7 kernel), while complex tampering types like copy-move and splice show greater sensitivity to the loss of high-frequency details (18.25% and 19.33% IoU drops respectively at 7 $\times$ 7 kernel).

Based on these findings, we propose several potential improvements for future research. These include incorporating distorted images during training to enhance model adaptation, developing more robust feature extraction mechanisms that preserve both local and global tampering evidence under distortions, and implementing adaptive detection thresholds based on estimated image quality. While implementing these improvements is beyond our current scope, they represent promising directions for enhancing the practical applicability of tampering detection systems.