Reproduce “FaceSigns: Semi-Fragile Neural Watermarks for Media Authentication and Countering Deepfakes” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Paarth Neekhara, Shehzeen Hussain, Xinqiao Zhang, Ke Huang, Julian McAuley, Farinaz Koushanfar
Researchers at the University of California San Diego introduced FaceSigns, a semi-fragile neural watermarking system for digital media authentication that embeds an imperceptible, verifiable secret message. The framework achieves high robustness to common image processing operations while rendering the watermark unrecoverable under malicious facial manipulations like Deepfakes, demonstrating a manipulation detection AUC of 0.996.
Key insights
- Explicitly formulating the message retrieval loss to minimize error for benignly processed images and maximize error for maliciously processed images enables the learning of selective fragility.
- Modeling malicious tampering as a localized attenuation of the watermark in facial regions, rather than relying on specific Deepfake models, allows the system to generalize detection to unseen Deepfake techniques.
- Utilizing a U-Net architecture for the encoder/decoder with specific upsampling methods and combining perceptual and adversarial losses contributes to higher imperceptibility of the embedded watermark compared to prior methods.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.