Reproduce “GReaTer: Gradients over Reasoning Makes Smaller Language Models Strong Prompt Optimizers” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Sarkar Snigdha Sarathi Das, Ryo Kamoi, Bo Pang, Yusen Zhang, Caiming Xiong, Rui Zhang

A new prompt optimization method called GREATER empowers smaller, open-source language models to self-optimize their prompts by leveraging numerical gradients derived from the model's entire reasoning process. This technique enables lightweight models to achieve performance comparable to or better than prompts optimized by massive proprietary LLMs, showing an average 8.9% improvement over state-of-the-art methods on Big-Bench Hard tasks.

Key insights

  • Numerical gradients, when computed over the entire reasoning chain of an LLM, provide a more direct and effective signal for discrete prompt optimization than textual feedback.
  • Smaller, open-source language models can achieve high-quality prompt optimization without relying on massive, proprietary LLMs for feedback.
  • The generated prompts offer structured, innovative, and task-specific guidance for problem-solving, going beyond generic instructions.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.