Reproduce “Conflict-Aware Additive Guidance for Flow Models under Compositional Rewards” on OpenResearch

About this paper

Xuehui Yu, Fucheng Cai, Meiyi Wang, Xiaopeng Fan, Harold Soh

Researchers at the National University of Singapore and Harbin Institute of Technology developed Conflict-Aware Additive Guidance (g_car) to mitigate off-manifold drift in continuous-time flow models under compositional reward settings. The method dynamically resolves gradient conflicts, significantly improving constraint satisfaction and sample quality across diverse tasks, including generative decision-making and text-guided image manipulation, with a modest computational overhead.

Key insights

• A theoretical analysis identifies gradient misalignment as a critical factor contributing to approximation error and off-manifold drift in guided sampling, showing its severe scaling with the number of reward functions and angular divergence.
• Approximation error in guided sampling can be decomposed into coupling shift, gradient misalignment, and localized approximation errors, providing a principled understanding of off-manifold drift.
• By selectively activating a learnable, conservative corrective vector field only when significant gradient conflicts are detected, the system can effectively rectify off-manifold drift while maintaining computational efficiency.

Summary by alphaXiv.