Reproduce “sketch-plot: Progressive Editing for Text-to-Image Academic Figures” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yinghao Tang, Yupeng Xie, Yingchaojie Feng, Tingfeng Lan, Jiale Lao, Wei Chen
The sketch-plot system enables progressive editing of academic figures generated by text-to-image models, transforming flat raster images into structured, editable compositions. It segments figures into semantically addressable "puzzle pieces" and provides multi-layer control, including on-demand vectorization, to enhance post-generation refinement and efficiency.
Key insights
- The "puzzle paradigm," which ensures an exact tiling of the canvas with addressable pieces, makes imperfect VLM layout outputs functional by providing a complete and consistently editable surface for user interaction.
- A multi-layer progressive editing workflow allows users to apply the appropriate level of control (raster, segmented raster, vector) as needed, optimizing for computational cost and interaction fidelity.
- Integrating a human-in-the-loop design effectively mitigates the limitations of current automated segmentation and vectorization models for complex academic figures, fostering robust human-AI collaboration.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.