Reproduce “MuSC: Improving Complex Instruction Following with Multi-granularity Self-Contrastive Training” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Hui Huang, Jiaheng Liu, Yancheng He, Shilong Li, Bing Xu, Conghui Zhu, Muyun Yang, Tiejun Zhao
A multi-granularity self-contrastive training framework, MuSC, enables Large Language Models to more accurately follow complex instructions, outperforming existing self-alignment methods on benchmarks like CFBench and FollowBench. The approach avoids reliance on stronger external models by generating its own preference data and optimizing at both constraint and token levels, demonstrating scalability across model sizes.
Key insights
- Effective contrastive learning for complex instructions can be achieved by generating preference data that explicitly highlights differences in constraint fulfillment at a coarse, instruction-level granularity.
- Self-supervised, token-level optimization guided by calibrated model confidence (entropy) significantly enhances learning efficiency and precision for aligning with subtle instruction nuances.
- A carefully designed negative sampling scheme, such as constraint dropout, can create optimal 'rejected' responses that maintain a rational edit distance from 'chosen' responses, fostering more effective learning than overly divergent negative examples.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.