Reproduce “Deep Probabilistic Accelerated Evaluation: A Robust Certifiable Rare-Event Simulation Methodology for Black-Box Safety-Critical Systems” on OpenResearch

About this paper

Mansur Arief, Zhiyuan Huang, Guru Koushik Senthil Kumar, Yuanlu Bai, Shengyi He, Wenhao Ding, Henry Lam, Ding Zhao

The Deep Probabilistic Accelerated Evaluation (Deep-PrAE) framework provides a robust and certifiable methodology for estimating rare-event probabilities in black-box safety-critical systems. It introduces a "relaxed efficiency certificate" to guarantee reliable upper and lower bounds on risk, addressing the issue of undetected under-estimation inherent in many existing black-box simulation techniques.

Key insights

• Traditional black-box Importance Sampling methods can severely and *undetectably* underestimate rare-event probabilities, even when their empirical relative error appears low.
• Under the assumption of "orthogonal monotonicity" for the rare-event set, deep neural networks can be trained to learn guaranteed one-sided approximations (outer for upper bounds, inner for lower bounds) of the true rare-event set.
• The concept of a "relaxed efficiency certificate" allows for robust statistical guarantees in black-box systems by providing bounds on the rare-event probability, which is more practical than precise point estimation.

Summary by alphaXiv.