Reproduce “NagaNLP: Bootstrapping NLP for Low-Resource Nagamese Creole with Human-in-the-Loop Synthetic Data” on OpenResearch

About this paper

Agniva Maiti, Manya Pandey, Murari Mandal

Researchers from RespAI Lab at KIIT Bhubaneswar developed an LLM-to-human bootstrapping pipeline to create extensive NLP resources for Nagamese Creole, a language with minimal digital footprint. This method generated the NagaNLP toolkit, enabling new state-of-the-art benchmarks for POS tagging (93.81% accuracy) and NER (0.75 macro F1-score), and significantly improved the perplexity of a fine-tuned Llama 3.2 model to 3.85 for generative tasks.

Key insights

• Large Language Models can effectively serve as "language elicitors" for zero-resource languages, developing an internal linguistic representation through guided, iterative interaction with human experts.
• Human-in-the-loop validation is critical for quality control, significantly improving performance (e.g., 9-13 F1-point gains) of models trained on synthetic data by ensuring linguistic authenticity.
• Instruction-tuning state-of-the-art generative models on carefully curated, synthetically generated, and human-validated data is highly effective for low-resource languages, outperforming proxy-language baselines.

Summary by alphaXiv.