Reproduce “BlenderLLM: Training Large Language Models for Computer-Aided Design with Self-improvement” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Yuhao Du, Shunian Chen, Wenbo Zan, Peizhao Li, Mingxuan Wang, Dingjie Song, Bo Li, Yan Hu and 1 more

BlenderLLM introduces a comprehensive framework for training Large Language Models to generate Blender Python scripts from natural language instructions. The model, trained on the newly created BlendNet dataset with a self-improvement strategy, achieves state-of-the-art performance and exhibits a remarkably low syntax error rate in generating 3D models from diverse instructions, as evaluated by the new CADBench benchmark.

Key insights

  • High-quality, domain-specific datasets (BlendNet) and robust evaluation benchmarks (CADBench) are crucial for successfully adapting Large Language Models to complex applications like CAD.
  • Self-improvement, an iterative process of generating and filtering data to fine-tune the model, effectively mitigates data scarcity and significantly enhances LLM performance in specialized domains like CAD, particularly for spatial reasoning and instruction following.
  • Open-source LLMs can achieve state-of-the-art performance in niche domains when combined with targeted fine-tuning and self-improvement strategies, even surpassing larger closed-source models in specific metrics.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.