Reproduce “Adaptive Smooth Tchebycheff Attention for Multi-Objective Policy Optimization” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Alejandro Murillo-Gonzalez, Mahmoud Ali, Lantao Liu
Indiana University–Bloomington researchers developed PASTA (Policy-optimization via Adaptive Smooth Tchebycheff Attention), an algorithm that dynamically adjusts the smoothness of Tchebycheff scalarization to robustly discover Pareto-optimal policies in multi-objective reinforcement learning, particularly in non-convex regions. The approach achieved a 45.5% Hypervolume improvement over baselines in a real-world robotic stealth visual search task.
Key insights
- Dynamically adjusting the Smooth Tchebycheff (STCH) parameter (μ) based on real-time gradient conflict enables robust discovery of non-convex Pareto-optimal policies while maintaining optimization stability.
- The conflict ratio derived from gradient projection (PCGrad) serves as an effective feedback signal for adaptively controlling the scalarization's smoothness.
- Aligning the critic's optimization with the actor's scalarization, via attention-weighted loss in a branched critic, is crucial for consistent and effective multi-objective policy improvement.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.