Reproduce “Prompting Frameworks for Large Language Models: A Survey” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Xiaoxia Liu, Jingyi Wang, Jun Sun, Xiaohan Yuan, Guoliang Dong, Peng Di, Wenhai Wang, Dongxia Wang

This survey provides a comprehensive overview of prompting frameworks for large language models, categorizing them into three types based on their primary focus: LLM Shells, LLM Interaction Languages, and LLM Output Restrictors. The work details their architecture, capabilities, and challenges, identifying common features and current limitations.

Key insights

  • Prompting frameworks can be categorized into three main classes based on their primary focus: The Shell of LLMs (LLM-SH) for integration and workflow orchestration, Language for Interaction with LLMs (LLM-LNG) for structured control via specialized syntax, and Output Restrictors of LLMs (LLM-RSTR) for enforcing content and structural constraints.
  • LLM-SH frameworks generally offer the most comprehensive feature sets, including memory capabilities and external tool integration, while LLM-LNG and LLM-RSTR frameworks excel in their specialized areas but often have fewer general-purpose features.
  • Despite their benefits, prompting frameworks face challenges such as security vulnerabilities (e.g., prompt injection), technical limitations (e.g., inconsistent LLM integration), and adoption barriers (e.g., steep learning curves).

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.