Reproduce “Order-Disorder: Imitation Adversarial Attacks for Black-box Neural Ranking Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jiawei Liu, Yangyang Kang, Di Tang, Kaisong Song, Changlong Sun, Xiaofeng Wang, Wei Lu, Xiaozhong Liu
An imitation adversarial attack method was developed for black-box neural text ranking models, demonstrating that irrelevant passages can be boosted in rank by stealthy "triggers." The approach achieved over 90% attack success rates on models like MiniLM while maintaining high imperceptibility to human evaluators and evading automated detection systems.
Key insights
- Black-box neural ranking models can be accurately imitated through knowledge distillation, leveraging pseudo-relevance feedback from observed rankings to create functionally similar models that achieve comparable performance to the target.
- Adversarial triggers generated on these imitation models exhibit strong transferability, effectively manipulating the ranking behavior of the original black-box victim models.
- Incorporating linguistic constraints (fluency and semantic consistency) is critical for producing stealthy adversarial triggers that are difficult for humans and automated systems to detect, highlighting a trade-off between attack strength and imperceptibility.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.