Reproduce “SlideAudit: A Dataset and Taxonomy for Automated Evaluation of Presentation Slides” on OpenResearch

About this paper

Zhuohao Jerry Zhang, Ruiqi Chen, Mingyuan Zhong, Jacob O. Wobbrock

A research team from the University of Washington developed SlideAudit, the first richly-annotated dataset and a systematic taxonomy of 27 categories across five dimensions for identifying design flaws in presentation slides. Their evaluations demonstrate that providing large language models (LLMs) with this taxonomy significantly improves their ability to detect slide design flaws (up to an F1 score of 0.655 for GPT-4o) and generate effective, actionable remediation plans, outperforming prior general design critique frameworks.

Key insights

• Providing LLMs with a structured, domain-specific taxonomy significantly improves their performance in identifying presentation slide design flaws and generating higher quality, more executable remediation plans, showing a clear advantage over zero-shot prompting or generic design frameworks.
• Computational augmentations (e.g., saliency maps, color metrics) did not further enhance LLM performance for flaw detection, suggesting that LLMs might not effectively integrate such signals or could be misled by them in complex design contexts.
• SlideAudit's compact, machine-actionable taxonomy is more effective for automated evaluation than broad, human-centered principles or extensive free-text critiques from prior work, demonstrating the value of domain-specific structure.

Summary by alphaXiv.