Reproduce “Meta-Black-Box-Optimization through Offline Q-function Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zeyuan Ma, Zhiguang Cao, Zhou Jiang, Hongshu Guo, Yue-Jiao Gong
Q-Mamba introduces an offline Meta-Black-Box-Optimization framework that addresses the inefficiency and action space complexity of online reinforcement learning approaches. It utilizes a decomposed Q-function and the Mamba architecture to effectively learn algorithm configurations from pre-collected data, demonstrating competitive or superior optimization performance on benchmark functions and neuroevolution tasks while significantly reducing training costs.
Key insights
- Decomposing the Q-function into a sequence of smaller action spaces effectively addresses the challenge of high-dimensional joint action spaces in algorithm configuration.
- The Mamba architecture's efficient handling of long sequences and selective state space modeling is well-suited for the dynamic, sequential decision-making in MetaBBO.
- A balanced offline dataset, combining exploitation and exploration trajectories, along with conservative Q-learning, helps mitigate distributional shift and ensures robust learning in offline MetaBBO.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.