Reproduce “Implementation Risk in Portfolio Backtesting: A Previously Unquantified Source of Error” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Dong Yin, Takeshi Miki, Vladislav Lesnichenko, Vasyl Gural
A study introduced and quantified 'implementation risk' in portfolio backtesting, demonstrating that the choice of simulation engine can cause divergence in reported strategy performance despite identical inputs. It revealed that high-turnover, cost-intensive strategies experienced total return differences up to 3.71% across engines, with simple strategies showing negligible variation, advocating for external validation of backtesting software.
Key insights
- Implementation risk is a material and quantifiable source of error in portfolio backtesting, distinct from statistical overfitting, primarily driven by differences in transaction cost modeling.
- The magnitude of divergence in backtest outcomes scales monotonically with strategy complexity and cost intensity, with high-turnover strategies exhibiting the largest discrepancies.
- Multi-engine comparison serves as a powerful diagnostic tool for uncovering critical, otherwise undetected bugs, misconfigurations, and fundamental design flaws within individual backtesting engines.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.