Reproduce “SymboLLM-FE: LLM-Accelerated Symbolic Regression for Automated Feature Engineering on Tabular Data” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zi-Jian Cheng, Zi-Yi Jia, Zhi Zhou, Yu-Feng Li, Lan-Zhe Guo
Researchers from Nanjing University developed SymboLLM-FE, a two-stage framework that integrates symbolic regression and large language models (LLMs) for automated feature engineering on tabular data. This approach generates high-performing, interpretable features with significantly improved computational efficiency, demonstrating an average 1.23% gain over traditional AutoFE methods and approximately 1% higher accuracy than existing LLM-based approaches while requiring far fewer LLM API calls.
Key insights
- Combining the mathematical rigor and efficiency of symbolic regression with the semantic reasoning capabilities of LLMs effectively bridges the gap between predictive performance and feature interpretability.
- Constraining LLM operations to a refinement role, grounded in statistically validated symbolic regression rules, drastically mitigates the risks of hallucination and implicit bias while significantly reducing computational costs and API calls.
- Efficiently sampling the feature search space using Spearman correlation and an expanding-sliding window is crucial for the scalability and practical application of symbolic regression in high-dimensional tabular data.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.