Reproduce “cadrille: Multi-modal CAD Reconstruction with Reinforcement Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Maksim Kolodiazhnyi, Denis Tarasov, Dmitrii Zhemchuzhnikov, Alexander Nikulin, Ilya Zisman, Anna Vorontsova, Anton Konushin, Vladislav Kurenkov and 1 more

CADRILLE introduces a unified multimodal framework for reconstructing CAD models from point clouds, images, or text, leveraging a two-stage supervised and reinforcement learning pipeline. The model generates executable Python code, achieving state-of-the-art accuracy and significantly reducing invalid code generation across multiple benchmarks and real-world datasets.

Key insights

  • A unified VLM-based architecture can effectively process diverse input modalities and generate expressive programmatic CAD representations.
  • The two-stage training approach, especially RL fine-tuning on handcrafted data, effectively bridges the domain gap between synthetic and real-world data, significantly improving cross-dataset generalization.
  • Reinforcement learning, guided by 3D geometric rewards, dramatically enhances the robustness and validity of generated CAD code without requiring ground truth CAD sequences for the RL phase.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.