Reproduce “Controllable Abstraction in Summary Generation for Large Language Models via Prompt Engineering” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Xiangchen Song, Yuchen Liu, Yaxuan Luan, Jinxu Guo, Xiaofan Guo
Researchers developed a multi-stage prompt engineering framework that enables large language models to produce summaries with controllable abstraction levels. This method achieved ROUGE-N of 0.50, ROUGE-L of 0.46, and BLEU of 0.45 on the CNN/Daily Mail dataset, outperforming existing baselines.
Key insights
- Integrating semantic graph construction and a multi-level optimization strategy within a prompt engineering framework enhances summary accuracy and fluency.
- There is an optimal prompt length (30-40 tokens) that maximizes summary quality by providing sufficient context without introducing redundancy.
- Summary generation performance is influenced by text characteristics, with news articles yielding better results than blog or academic texts due to their structured nature.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.