Reproduce “Coprocessor Actor Critic: A Model-Based Reinforcement Learning Approach For Adaptive Brain Stimulation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Michelle Pan, Mariah Schrum, Vivek Myers, Erdem Bıyık, Anca Dragan
Researchers at UC Berkeley and USC developed Coprocessor Actor Critic (CopAC), a model-based reinforcement learning method for adaptive brain stimulation. This approach learns effective neural coprocessor policies with over 10x fewer online interactions and higher task success compared to baselines, even achieving high performance during the learning phase.
Key insights
- Decomposing the learning problem by pre-training general world-action values offline and focusing online learning on patient-specific brain models significantly improves sample efficiency.
- Guiding online exploration using the pre-trained world-action value function enables the system to sample high-value actions, leading to faster learning and higher rewards during training.
- Iterative recalibration of the world-action value model against the capabilities of the injured brain prevents over-optimism and ensures the learned policy is tailored to what the patient can realistically achieve.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.