Reproduce “2ndMatch: Finetuning Pruned Diffusion Models via Second-Order Jacobian Matching” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Caleb Zheng, Eli Shlizerman
The 2ndMatch framework proposes a finetuning objective for pruned diffusion models, incorporating a second-order Jacobian matching loss to align the model's temporal sensitivity with a dense teacher. This approach leads to improved generative quality across architectures and pruning ratios, significantly narrowing the performance gap with unpruned models.
Key insights
- First-order Jacobian matching offers limited additional benefit for diffusion models beyond output-level knowledge distillation due to their inherent stochasticity and noisy inputs.
- Aligning the second-order Jacobian metric ($J^T J$) between student and teacher effectively transfers the dynamic stability and temporal sensitivity of the denoising trajectory.
- Approximating multi-step Jacobian propagation with a local, per-timestep second-order sensitivity proxy, through random projections, is crucial for scalable implementation.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.