Reproduce “Beyond Visual Evidence: Revealing and Mitigating Relational Privacy Leakage in Document MLLMs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Beining Xu, Hairui Wang, Jiaxin Wang, Changsheng Chen, Anirban Chakraborty
Researchers introduced relational privacy leakage, a vulnerability in document MLLMs where models jointly disclose correlated sensitive fields under weak or absent visual evidence, and developed DRUF, a framework that reduces this leakage to near zero while maintaining high Key Information Extraction utility.
Key insights
- Document MLLMs are susceptible to 'relational privacy leakage,' where they rely on memorized correlations between sensitive fields to jointly disclose personal information when visual evidence is insufficient.
- The quality of training data and the type of input (e.g., blank images, irrelevant photos) significantly influence the extent and nature of relational privacy leakage in MLLMs.
- Dynamically identifying and targeting sensitive field pairs for unlearning, rather than individual fields or entire samples, effectively mitigates relational leakage while preserving the model's core task capabilities.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.