Reproduce “Improving Object Detection by Modifying Synthetic Data with Explainable AI” on OpenResearch

About this paper

Nitish Mital, Simon Malzard, Richard Walters, Celso M. De Melo, Raghuveer Rao, Victoria Nockles

Researchers from DARe, The Alan Turing Institute, and DEVCOM Army Research Laboratory developed an Explainable AI (XAI)-guided, human-in-the-loop framework to iteratively modify 3D synthetic data, improving object detection performance. This method, applied to infrared vehicle detection, demonstrates that both increasing and decreasing realism in synthetic data can enhance model robustness, leading to a 6.1% mAP50 improvement over a real-data-only baseline.

Key insights

• Effective synthetic data modification can involve both increasing realism (to reinforce unique features) and decreasing realism (to disrupt common, confusing features), moving beyond a simplistic pursuit of photorealism.
• Explainable AI provides precise, interpretable guidance to human operators, making synthetic data curation more efficient and less reliant on intuition.
• The approach is model-agnostic and directly influences model behavior by targeted changes to the synthetic training data itself, rather than solely through algorithm optimization.

Summary by alphaXiv.