Enhancing Cross-Dataset Generalization in Deepfake Detection: A Multi-Source Fine-Tuning Approach with XceptionNet

The proliferation of generative models has enabled increasingly realistic deepfake content, posing serious risks to digital trust and public information integrity. While convolutional neural networks such as XceptionNet achieve near-perfect accuracy on intra-dataset deepfake benchmarks, their performance typically deteriorates on samples drawn from unseen distributions, a phenomenon known as the generalization gap. This study proposes a Multi-Source Fine-Tuning (MSFT) pipeline designed to mitigate dataset-specific overfitting in deepfake forensics. An ImageNet-pretrained XceptionNet backbone is adapted across two distinct distributions, namely a GAN-based facial image dataset of 140k real and fake faces and a combined FaceForensics++ and Celeb-DF video-frame distribution, through a two-phase training regimen: feature extraction with a frozen backbone, followed by end-to-end fine-tuning at an ultra-low learning rate to mitigate catastrophic forgetting. The classification head replaces the original ImageNet layers with Global Average Pooling, two regularized dense layers, and a sigmoid output. Evaluation on a held-out subset of 2,400 samples, using a strict authenticity confidence threshold of 0.70, yielded an overall accuracy of 87%, a recall of 96% on manipulated media, and only 43 false positives, demonstrating the pipeline’s suitability as a precision-oriented forensic filter. Qualitative analysis identifies heavy JPEG compression, low-light environments, and multi-face frames as the principal remaining failure modes. Accepted at the 2026 9th International Seminar on Research of Information Technology and Intelligent Systems (ISRITI) (Paper #1571336392).

Recommended citation: Yovan, K., Tarigan, G. A., & Sutoyo, R. (2026). "Enhancing Cross-Dataset Generalization in Deepfake Detection: A Multi-Source Fine-Tuning Approach with XceptionNet." 2026 9th International Seminar on Research of Information Technology and Intelligent Systems (ISRITI).