Reproduce “Which Hyperparameters Matter? A Game-Theoretic Framework for Interpretable Hyperparameter Sensitivity Analysis” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Nyi Nyi Aung, Heepeom Shin, Abigail Lawlor, Adrian Stein
A game-theoretic framework analyzes hyperparameter–objective interactions in machine learning models, utilizing Shapley Effects for global sensitivity analysis and Pareto front analysis for early model evaluation. This approach provides interpretable insights into hyperparameter influence, enabling more efficient optimization by identifying critical parameters and revealing performance trade-offs across various applications.
Key insights
- The influence of hyperparameters is highly application-dependent, with the same parameter exhibiting substantially different sensitivities across various models and tasks.
- Shapley Effects can effectively identify non-influential hyperparameters, allowing practitioners to reduce the search space and guide subsequent HPO efforts more strategically.
- Joint analysis of global sensitivity and Pareto fronts provides an early-stage model evaluation capability, indicating a model's performance potential and inherent trade-offs before extensive fine-tuning.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.