Reproduce “KIT-19: A Comprehensive Korean Instruction Toolkit on 19 Tasks for Fine-Tuning Korean Large Language Models” on OpenResearch

About this paper

Dongjun Jang, Sungjoo Byun, Hyemi Jo, Hyopil Shin

Researchers at Seoul National University developed KIT-19, a comprehensive Korean instruction toolkit, by converting 19 native Korean NLP datasets into 100,000 instruction-following instances. This dataset enabled fine-tuned Korean LLMs to achieve an average of 15% higher performance across benchmarks compared to existing models, with smaller models also outperforming larger baselines.

Key insights

• High-quality, human-crafted instruction data, derived from existing native language resources, is crucial for developing robust and culturally sensitive LLMs in contexts with fewer available resources than English.
• Careful dataset design, including balanced task ratios and diverse templates, can significantly improve LLM performance and generalizability across a wide array of tasks, including unseen ones.
• The quality of instruction data can substantially boost the efficiency of smaller LLMs, enabling them to achieve performance competitive with or superior to larger models trained on less optimal data.

Summary by alphaXiv.