Reproduce “Learning Action Hierarchies via Hybrid Geometric Diffusion” on OpenResearch

About this paper

Arjun Ramesh Kaushik, Nalini K. Ratha, Venu Govindaraju

A diffusion-based framework called HybridTAS advances temporal action segmentation by integrating Euclidean and hyperbolic geometries to explicitly model action hierarchies. It achieves an average F1 score of 90.6 on the 50Salads dataset, representing a 2.1 point improvement over prior methods, and processes video with 66% fewer inference steps on GTEA while preserving performance.

Key insights

• Explicitly modeling action hierarchies within a diffusion model's latent space using hyperbolic geometry leads to improved semantic understanding and more accurate temporal action segmentation.
• A two-phase optimization strategy, which first stabilizes action prototypes in hyperbolic space and then guides the diffusion process towards these fixed targets, is crucial for robust hierarchical learning.
• Hybrid geometric guidance enables the diffusion model to achieve superior performance with fewer inference steps, demonstrating increased efficiency and faster convergence.

Summary by alphaXiv.