Reproduce “Unified token representations for sequential decision models” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhuojing Tian, Yushu Chen
This work introduces a Unified Token Representation (UTR) and associated architectural variants for sequential decision models, addressing computational inefficiency in Transformer-based offline reinforcement learning. The approach demonstrates improved efficiency with up to a 74.92% reduction in FLOPs and a 30.02% training time speedup, while maintaining or surpassing state-of-the-art performance on D4RL benchmarks.
Key insights
- Fusing trajectory components into a single token (UTR) fundamentally reduces the sequence length for Transformers, leading to substantial computational savings.
- A theoretical analysis using Rademacher complexity suggests that UTR provides a tighter generalization bound, implying improved generalization capacity.
- Gated CNNs are highly effective for sequence mixing in RL, offering a lightweight and efficient alternative to self-attention by leveraging local causal dependencies.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.