Reproduce “OSP-Next: Efficient High-Quality Video Generation with Sparse Sequence Parallelism, HiF8 Quantization, and Reinforcement Learning” on OpenResearch

About this paper

Yunyang Ge, Xianyi He, Zezhong Zhang, Bin Lin, Bin Zhu, Xinhua Cheng, Li Yuan

An integrated framework named OSP-Next, developed by the Open-Sora Plan Team at Peking University, Rabbitpre AI, and Nanyang Technological University, enables efficient and high-quality text-to-video generation. It combines novel Skiparse-2D Attention, Sparse Sequence Parallelism, HiF8 quantization, and RL post-training to achieve VBench scores comparable to full-attention models (83.73%) and up to 2.27x inference speedups while maintaining visual fidelity.

Key insights

• Designing sparse attention to explicitly account for 2D spatial locality and maintaining native compatibility with standard efficient kernels (FlashAttention) is crucial for both performance and efficiency in video generation.
• Tailoring parallelization strategies (SSP) to the specific characteristics of sparse attention patterns significantly reduces communication overhead and improves load balance, accelerating multi-device video generation.
• Combining an advanced 8-bit quantization format (HiF8) with a reinforcement learning post-training phase (Mix-GRPO) can effectively preserve and even enhance generative quality in highly optimized, efficient video models.

Summary by alphaXiv.