Reproduce “Retromorphic Testing: A New Approach to the Test Oracle Problem” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Boxi Yu, Qiuyang Mang, Qingshuo Guo, Pinjia He
A new black-box testing methodology, Retromorphic Testing, addresses the test oracle problem by using a dual-program structure to perform and then reverse data transformations. It is shown to detect inconsistencies across diverse software, from traditional algorithms to AI systems like machine translation, while also formalizing existing ad-hoc testing strategies like Pivoted Query Synthesis.
Key insights
- The core idea for test oracle construction is inspired by the mathematical property of inverse functions, enabling an auxiliary program to 'reverse' the transformation of the program under test.
- Retromorphic Testing offers a distinct black-box paradigm for oracle construction that complements existing methods like differential and metamorphic testing, capable of identifying different classes of bugs.
- The framework provides a unifying theoretical basis for existing ad-hoc testing solutions, such as Pivoted Query Synthesis for database systems, promoting a more systematic approach to test oracle design.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.