Reproduce “A Framework for Empowering Reinforcement Learning Agents with Causal Analysis: Enhancing Automated Cryptocurrency Trading” on OpenResearch

About this paper

Rasoul Amirzadeh, Dhananjay Thiruvady, Asef Nazari, Mong Shan Ee

CausalReinforceNet (CRN) is a reinforcement learning framework for automated cryptocurrency trading that integrates Bayesian Networks for causal feature engineering and Dynamic Bayesian Networks for price direction prediction. The CRN agents, specifically CRN_PPO and CRN_DDPG, generally outperform traditional buy-and-hold strategies and baseline RL models across five altcoins, achieving improved profitability and risk mitigation in volatile markets.

Key insights

• Integrating causal feature engineering through Bayesian Networks provides a richer and more relevant state representation for RL agents, leading to more informed and effective trading decisions than using basic features alone.
• Dynamic Bayesian Networks can provide crucial probabilistic forecasts of future price directions, which, when incorporated into an RL agent's observations, enhance its ability to adapt to market volatility and mitigate risk.
• Reinforcement Learning agents, when augmented with sophisticated causal analysis, can develop adaptive and risk-aware trading strategies that consistently outperform passive investment and simpler RL models in volatile cryptocurrency markets.

Summary by alphaXiv.