Reproduce “A General Framework on Enhancing Portfolio Management with Reinforcement Learning” on OpenResearch

About this paper

Yinheng Li, Junhao Wang, Yijie Cao

A study from Columbia University introduces a comprehensive reinforcement learning framework for portfolio management that integrates real-world constraints such as continuous asset weights, short selling, and transaction costs. It demonstrates that the Proximal Policy Optimization (PPO) algorithm combined with a Convolutional Neural Network (CNN) achieves the highest average portfolio value of 1.044 and the lowest maximum drawdown of 0.30% in simulated environments, outperforming traditional baselines and other RL approaches.

Key insights

• Integrating practical constraints like transaction costs and short selling capabilities into RL frameworks significantly enhances their realism and potential applicability in financial markets.
• Convolutional Neural Networks (CNNs) generally outperform Long Short-Term Memory (LSTM) networks for extracting relevant features from multi-feature time-series financial data within the proposed framework.
• Gradient-free optimization methods like Evolution Strategies (ES) show promising performance with simpler architectures, offering a viable alternative when computational resources or gradient complexity are concerns.

Summary by alphaXiv.