Reproduce “Modelling Hierarchical Structure between Dialogue Policy and Natural Language Generator with Option Framework for Task-oriented Dialogue System” on OpenResearch

About this paper

Jianhong Wang, Yuan Zhang, Tae-Kyun Kim, Yunjie Gu

This paper presents HDNO, a hierarchical reinforcement learning framework using the option framework to model the interaction between dialogue policy and a natural language generator in task-oriented dialogue systems. The approach integrates a discriminator-based reward to enhance comprehensibility while learning latent dialogue acts, achieving improved task success and natural language fluency on the MultiWOZ datasets.

Key insights

• Asynchronous updates between high-level dialogue policies and low-level natural language generators are crucial for stable training and guaranteed convergence in hierarchical reinforcement learning for dialogue systems.
• Leveraging latent dialogue acts within an option framework allows for interpretable decision-making in E2E dialogue systems without requiring laborious manual dialogue act design.
• Integrating a language model-based discriminator reward effectively mitigates the comprehensibility degradation often observed in reinforcement learning-trained dialogue systems, leading to more natural responses.

Summary by alphaXiv.