Reproduce “Pure interaction effects unseen by Random Forests” on OpenResearch

About this paper

Ricardo Blum, Munir Hiabu, Enno Mammen, Joseph Theo Meyer

Greedy one-variable splitting can leave Random Forests unable to detect effects that appear only through combinations of features. The study proves this failure mechanism for pure interactions and shows that forests using joint or randomized multi-step partitions can reduce prediction error, while offering no universal advantage on additive or hierarchical problems.

Key insights

• Random feature selection alone does not reliably expose pure interactions: even when an interaction variable is considered, a sequence of greedy one-dimensional splits may fail to create the required joint partition.
• Interaction-aware partitioning is most useful when strong synergistic effects compete with marginal signals; on additive and hierarchical models, conventional or extremely randomized forests can remain competitive.
• The choice of forest architecture matters more than ensemble averaging itself, but improved interaction detection comes with higher computational cost and does not guarantee better performance on every dataset.

Summary by alphaXiv.