Reproduce “Finding Support Examples for In-Context Learning” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Xiaonan Li, Xipeng Qiu

Xiaonan Li and Xipeng Qiu from Fudan University developed LENS (fiLter-thEN-Search), a two-stage method for systematically identifying highly effective "support examples" for in-context learning (ICL). This approach significantly enhances the performance and stability of large language models on diverse text classification tasks by providing a principled way to select context examples.

Key insights

  • A significant gap exists between traditional coreset selection and ICL, necessitating ICL-specific methodologies for effective example selection.
  • Support examples selected by LENS exhibit substantially reduced sensitivity to example order compared to randomly sampled examples, suggesting a more robust task characterization.
  • Ground truth labels are crucial for LENS's support examples, contrasting with some prior findings for randomly sampled ICL examples.
  • Support examples identified using LENS demonstrate strong transferability across various language models (e.g., GPT2-L to GPT-Neo-2.7B), indicating they capture fundamental, model-agnostic task characteristics.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.