Reproduce “Layered Neural Atlases for Consistent Video Editing” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Yoni Kasten, Dolev Ofri, Oliver Wang, Tali Dekel
This research introduces Layered Neural Atlases, a method for consistent video editing that decomposes natural videos into editable 2D neural atlases for foreground and background layers using coordinate-based MLPs. The approach ensures temporal consistency of edits across frames, allowing non-professional users to apply complex visual effects to dynamic videos while preserving scene effects like occlusions and shadows.
Key insights
- Decomposing videos into continuous, layered 2D neural atlases provides an intuitive and unified representation that inherently ensures temporal consistency for edits by mapping corresponding video pixels to the same atlas coordinates over time.
- Regularization terms like rigidity, optical flow, and sparsity are critical for generating semantically meaningful, interpretable, and editable atlases, even if they do not always drastically improve raw video reconstruction quality.
- Coordinate-based MLPs offer advantages over discrete atlas representations for video decomposition, yielding cleaner, higher-quality atlases due to their fully differentiable nature and implicit regularization properties.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.