Reproduce “ABC: Adversarial Behavioral Cloning for Offline Mode-Seeking Imitation Learning” on OpenResearch

About this paper

Eddy Hudson, Ishan Durugkar, Garrett Warnell, Peter Stone

Adversarial Behavioral Cloning (ABC) replaces Gaussian maximum-likelihood imitation with an offline discriminator-based objective that favors expert-like action modes rather than their average. In a multimodal 2D navigation task, ABC reaches an average reward of 758 versus 0.22 for behavioral cloning on combined-route demonstrations, and on Hopper data mixed with random actions it achieves 527 versus 10 for standard cloning, while performing similarly on clean expert data.

Key insights

• Mode-seeking is preferable to mean-seeking when several actions are valid but their average is not; in a bandit example, ABC moves toward one of the reward modes while BC predicts approximately -0.07 near the low-reward center.
• ABC is most useful when demonstrations are multimodal or contain outlier actions, rather than when the expert dataset is clean and approximately unimodal.
• The method remains tied to the states represented in the demonstrations, and its behavior depends on discriminator training, replay-buffer construction, and adversarial optimization.

Summary by alphaXiv.