Reproduce “DesignCoder: Hierarchy-Aware and Self-Correcting UI Code Generation with Large Language Models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yunnong Chen, Shixian Ding, YingYing Zhang, Wenkai Chen, Jinzhou Du, Lingyun Sun, Liuqing Chen
DesignCoder, developed by Zhejiang University in collaboration with Huawei, presents a framework that leverages Multimodal Large Language Models for generating React Native UI code from design mockups, integrating hierarchy-aware processing and a self-correcting mechanism. The system produces high-fidelity, structurally sound, and maintainable UI code, outperforming prior methods in visual and structural similarity metrics and receiving higher usability ratings from professional developers.
Key insights
- Decomposing complex UI understanding into visual division, semantic extraction, and hierarchical grouping significantly improves MLLM performance in interpreting intricate UI structures.
- Integrating design metadata and explicit hierarchy-aware processing is critical for generating UI code that maintains high visual fidelity and robust structural organization.
- An iterative, vision-aware self-correction mechanism is essential for detecting and rectifying visual and functional errors in MLLM-generated code, enhancing its reliability and production-readiness.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.