Reproduce “Computer-Aided Design as Language” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yaroslav Ganin, Sergey Bartunov, Yujia Li, Ethan Keller, Stefano Saliceti
DeepMind researchers developed a novel approach to automatically generate structured 2D Computer-Aided Design (CAD) sketches by treating design as a language. Their autoregressive Transformer models, particularly using a new 'triplet' data representation, learned to create plausible geometric entities and infer complex constraints directly from data, demonstrating strong performance in both unconditional generation and image-to-sketch translation.
Key insights
- Treating CAD sketch generation as a language modeling problem, by representing sketches as sequences of tokens, allows leveraging powerful autoregressive Transformer models.
- The proposed 'triplet' data representation, which combines discrete and quantized continuous values with explicit contextual information, consistently outperforms a raw byte representation in modeling CAD sketches, leading to more valid and solvable outputs.
- Integrating Pointer Networks within the Transformer architecture effectively handles complex relationships and references between geometric entities and constraints in a structured CAD environment.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.