Reproduce “On Investigating the Conservative Property of Score-Based Generative Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Chen-Hao Chao, Wei-Fang Sun, Bo-Wun Cheng, Chun-Yi Lee
The authors introduce quasi-conservative score-based models that keep a flexible score network while penalizing asymmetry in its Jacobian, associated with local rotation. On CIFAR-10, QC-NCSN++ uses 124 model evaluations and reaches FID 7.21, versus 170 evaluations and FID 7.48 for U-NCSN++ with an adaptive ODE sampler; these counts do not establish faster elapsed sampling, and training adds computation.
Key insights
- Empirical evidence demonstrates that non-conservative score fields, characterized by rotational components, lead to significantly less efficient sampling processes, requiring more steps to converge to the target distribution.
- While architectural flexibility in USBMs generally yields superior modeling performance (lower negative log-likelihood), this often comes at the cost of reduced conservativeness, which can degrade sampling quality.
- It is feasible to impart quasi-conservativeness into flexible SBMs through a tailored regularization loss, successfully combining the architectural advantages of USBMs with the mathematical robustness of CSBMs.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.