Reproduce “Weakly Supervised Disentangled Representation for Goal-conditioned Reinforcement Learning” on OpenResearch

About this paper

Zhifeng Qian, Mingyu You, Hongjun Zhou, Bin He

Researchers from Tongji University introduce DR-GRL, a framework that leverages a weakly supervised disentangled representation to improve visual goal-conditioned reinforcement learning. It enhances sample efficiency and policy generalization by learning object shape, color, and physically consistent position representations, demonstrating robust performance in simulated and real-world robotic pushing tasks.

Key insights

• A Spatial Transform module within the autoencoder can enforce consistency between the learned latent space and real physical positions, crucial for reliable reward signals and controllable goal generation.
• A progressive swap strategy, involving zero, one, and cycle swaps, effectively disentangles object attributes (shape, color, position) from weak supervision, enhancing representation learning without explicit labels.
• Combining disentangled, physically consistent representations with goal-conditioned RL and uniform relabeling significantly improves sample efficiency and generalization to unseen goals and objects.

Summary by alphaXiv.