Reproduce “Self-Compatibility: Evaluating Causal Discovery without Ground Truth” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Philipp M. Faller, Leena Chennuru Vankadara, Atalanti A. Mastakouri, Francesco Locatello, Dominik Janzing

The authors introduce Self-Compatibility, which checks causal-discovery outputs by comparing graphs learned from overlapping variable subsets. They define interventional and graphical scores for testing whether subset claims fit one joint causal model. For RCD, the interventional score correlated with SHD at 0.52 and selected a lower-or-equal-SHD setting in 72% of 100 simulated cases; compatibility does not guarantee correctness.

Key insights

  • Causal learning fundamentally requires stability across different subsets of variables, analogous to stability across data points in statistical learning theory.
  • Incompatibilities in causal models learned from different variable subsets can falsify an algorithm's output, indicating either violations of underlying assumptions or errors due to finite sample effects.
  • For certain "merging-enabling algorithms" like FCI, compatibility constraints can be strong enough to uniquely determine the underlying joint distribution from the marginal models.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.