Reproduce “Federated In-Context Learning: Iterative Refinement for Improved Answer Quality” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Ruhan Wang, Zhiyong Wang, Chengkai Huang, Rui Wang, Tong Yu, Lina Yao, John C.S. Lui, Dongruo Zhou
A collaborative effort from Indiana University, The Chinese University of Hong Kong, The University of New South Wales, CSIRO's Data61, and Adobe Research introduced Federated In-Context Learning (Fed-ICL), a privacy-preserving and communication-efficient framework that iteratively refines Large Language Model answers for Question Answering tasks. This approach leverages decentralized client data without transmitting raw information or model parameters, achieving strong performance across benchmarks like MMLU and TruthfulQA while incurring significantly lower communication costs than traditional federated methods.
Key insights
- Iterative collaborative refinement in a federated setup significantly enhances ICL performance, especially for less powerful LLMs, by aggregating decentralized knowledge over multiple rounds.
- Utilizing local client examples, even without ground truth labels, provides meaningful context that improves ICL answer quality and robustness, as demonstrated by the Fed-ICL-Free variant.
- The framework is supported by a theoretical analysis proving that the iterative update scheme converges to an optimal answer, providing a foundational understanding of its effectiveness.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.