Reproduce “VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Models” on OpenResearch

About this paper

Ajay Jain, Amber Xie, Pieter Abbeel

VectorFusion, developed by UC Berkeley, presents a method for generating high-quality, abstract, and editable vector graphics (SVGs) from text descriptions. It leverages pre-trained pixel-based diffusion models via Score Distillation Sampling and differentiable rasterization, outperforming previous text-to-vector approaches in quality and caption consistency across diverse artistic styles.

Key insights

• Pre-trained pixel-based diffusion models can be effectively abstracted and leveraged as generative priors for generating structured, non-pixel modalities like vector graphics, even without specialized vector datasets.
• Score Distillation Sampling (SDS) provides a powerful generative guidance signal that produces higher quality and more coherent vector outputs compared to discriminative (CLIP-based) optimization.
• Careful initialization and regularization techniques, such as periodic path reinitialization, are crucial for optimizing vector primitives and ensuring the utilization of all available paths for richer outputs.

Summary by alphaXiv.