Reproduce “QnRL: Quantum-Native Reinforcement Learning” on OpenResearch

About this paper

Alexander DeRieux, Walid Saad

A quantum-native reinforcement learning framework, QnRL, directly models stochastic environment behavior by representing return distributions as quantum states. It employs a novel Quantum Amplitude Kickback (QuAK) algorithm for in-quantum-space statistical analysis, demonstrating up to 94.3% fewer parameters and often superior evaluation performance compared to classical deep Q-network baselines across multiple control and game environments.

Key insights

• Operating entirely within quantum Hilbert space, the QuAK algorithm preserves information encoded in complex amplitudes and phases, addressing a critical limitation of hybrid quantum-classical approaches that suffer from quantum state collapse upon measurement.
• QnRL's ability to represent return distributions as complex quantum states, leveraging the phase dimension, provides enhanced expressiveness and allows for capturing complex correlations between observations, actions, and returns.
• For certain environments, considering higher-order distributional properties, such as the second moment (variance), proves more relevant for optimal policy learning than solely relying on the mean, a capability directly facilitated by the QuAK algorithm.

Summary by alphaXiv.