Reproduce “TrackTok: Object-Centric Video Tokenization with Semantically Persistent Tokens” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Nhi Pham, Christopher Wewer, Bernt Schiele, Jonas Fischer, Jan Eric Lenssen
TrackTok lets video tokens follow semantic content across frames rather than fixed image locations. It uses clustered features to initialize queries that produce compact latent representations. On UCF-101 generation, 128 tokens give gFVD 126, versus 102 for LARP using 1,024 tokens; at 32 tokens TrackTok scores 201 versus 672 for LARP. LARP and AdapTok score better at 1,024 tokens.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.