Reproduce “Evaluating GFlowNet from partial episodes for stable and flexible policy-based training” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Puhua Niu, Shili Wu, Xiaoning Qian
A novel Subtrajectory Evaluation Balance (Sub-EB) objective enhances the stability and flexibility of policy-based Generative Flow Networks (GFlowNets) by enabling more reliable learning of the evaluation function, V(s). This method demonstrates improved or competitive performance across various combinatorial generation tasks, including hypergrids, sequence design, and molecular graph design.
Key insights
- For a fixed forward policy "pi_F", the solution to the flow balance condition for log F(s) (from value-based principles) uniquely corresponds to the exact KL divergence from state s, which is the true evaluation function V(s).
- This theoretical unification allows adapting flow balance conditions, traditionally used for learning state flows, to reliably learn the evaluation function V(s) within the policy-based GFlowNet framework.
- The subtrajectory-wise loss in Sub-EB provides a more robust and flexible mechanism for estimating V(s) by incorporating information from complete subtrajectories, unlike traditional TD methods focusing on single-step transitions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.