Reproduce “Sandwich-Residuals: Parameter-Efficient Test-time Adaptation of World Models” on OpenResearch

About this paper

Krishnam Soni, Aditya Sehgal, Vedant Dave, Elmar Rueckert

Sandwich-Residuals adapts a pretrained world model to test-time shifts by learning small residual corrections before and after its frozen predictor. It updates 99,520 parameters online from self-supervised prediction errors, without rewards, labels, or source-domain data. Across 21 simulated conditions, it achieves 65.0% mean success, below the strongest internal-update variant at 68.2%, while using fewer adapted parameters.

Key insights

• Input-side action and visual corrections and output-side prediction corrections address different forms of mismatch; their relative value depends on whether the shift changes dynamics, perception, or both.
• Interface adaptation can approach the performance of selective internal fine-tuning without identifying an adaptation target: it reaches 65.0% average success versus 68.2% for the strongest AdaJEPA variant while using 97–99% fewer updated parameters.
• The approach works best when the frozen representation still preserves task-relevant information; severe visual changes that disrupt the encoder remain difficult to correct downstream.

Summary by alphaXiv.