Reproduce “Brewing Knowledge in Context: Distillation Perspectives on In-Context Learning” on OpenResearch

About this paper

Chengye Li, Haiyun Liu, Yuanxi Li

An analysis by researchers establishes In-Context Learning (ICL) in large language models as an implicit knowledge distillation process during inference. This framework demonstrates how demonstration tokens distill pre-trained knowledge into a task-specific reference model, revealing that generalization error decreases with more examples and that prompt-target distribution alignment minimizes bias.

Key insights

• ICL's ability to adapt to new tasks without parameter updates can be mechanistically understood as an implicit knowledge distillation, where demonstration tokens guide the formation of a compact, task-specific "reference model."
• The softmax normalization constant in attention functions as an inverse learning rate for this implicit distillation, influencing the 'aggressiveness' of knowledge transfer from the pre-trained model.
• The effectiveness of ICL is directly linked to the alignment between the prompt's demonstration distribution and the true target task distribution, quantifiable through Maximum Mean Discrepancy (MMD).

Summary by alphaXiv.