Reproduce “Don't Just Say "I don't know"! Self-aligning Large Language Models for Responding to Unknown Questions with Explanations” on OpenResearch

About this paper

Yang Deng, Yong Zhao, Moxin Li, See-Kiong Ng, Tat-Seng Chua

The research introduces Self-Aligned, a method that enables Large Language Models to identify unknown questions and generate explanatory responses for their unanswerability. This approach, using self-augmented and self-curated data, significantly improves unknown question detection, classification (e.g., 300-400% F1 increase for Vicuna), and open-ended explanatory response generation, while maintaining performance on known questions.

Key insights

• Providing explanations for unanswerability, beyond mere refusal or classification, is crucial for improving LLM trustworthiness, reliability, and user experience.
• Disparity-driven self-curation is an effective technique for generating high-quality training data for complex LLM behaviors like explaining unanswerability, minimizing reliance on extensive manual annotations.
• LLMs can be intrinsically aligned to communicate their knowledge boundaries and explain query limitations, leading to more transparent and helpful interactions.

Summary by alphaXiv.