Reproduce “Feature-Weighted Linear Stacking” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Joseph Sill, Gabor Takacs, Lester Mackey, David Lin
Feature-Weighted Linear Stacking (FWLS) is a linear ensemble method that adaptively blends predictions from multiple base models by weighting them as linear functions of meta-features. This approach achieved a 0.001972 RMSE reduction on the Netflix Prize test set compared to standard linear stacking, demonstrating that context-dependent model blending can be effectively achieved with linear techniques.
Key insights
- Linear models can effectively incorporate meta-features for adaptive blending without requiring complex, nonlinear, or iterative algorithms.
- The interaction between base model predictions and meta-features is crucial for accuracy gains; simply adding meta-features as linear inputs is insufficient.
- The Feature-Weighted Linear Stacking (FWLS) approach retains the virtues of linear models, including computational speed, stability, and interpretability, while achieving significant accuracy improvements.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.