Reproduce “A Bayesian Framework of Deep Reinforcement Learning for Joint O-RAN/MEC Orchestration” on OpenResearch

About this paper

Fahri Wisnu Murti, Samad Ali, Matti Latva-aho

Researchers at the University of Oulu introduced a Bayesian deep reinforcement learning framework for joint orchestration of Open Radio Access Network (O-RAN) and Multi-access Edge Computing (MEC) resources. The Bayesian BDDQN algorithm optimizes functional split selection, resource allocation, component placement, and traffic routing, reducing MEC service delay costs by up to 52.83% and outperforming non-Bayesian counterparts by up to 32% in accumulated reward.

Key insights

• Decomposing the multi-dimensional action space into sub-action spaces through a branching Q-network effectively manages the complexity of joint O-RAN/MEC orchestration decisions.
• Integrating a Bayesian framework (Gaussian Bayesian Linear Regression) into the Q-network's output layer enables uncertainty quantification, leading to improved exploration via Thompson sampling and better data efficiency.
• A model-free deep reinforcement learning approach is well-suited for dynamically orchestrating complex softwarized networks, as it avoids the need for explicit and often hard-to-obtain system models.

Summary by alphaXiv.