Reproduce “MOTOR: A Multimodal Dataset for Two-Wheeler Rider Behavior Understanding” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Varun A. Paturkar, Shankar Gangisetty, C.V. Jawahar
The MOTOR dataset is introduced as a large-scale, multimodal resource containing 25+ hours of synchronized video, eye-gaze, audio, and telemetry data from two-wheeler riders in various traffic scenarios. It provides rich annotations for 12 rider behaviors and their legality, enabling benchmarks for behavior recognition and legality classification, with transformer-based models achieving 52.9% accuracy for behavior and 69.0% for legality when utilizing multimodal fusion.
Key insights
- Multimodal fusion of ego-vehicle video, rider eye-gaze, and telemetry data significantly improves the accuracy of two-wheeler rider behavior recognition and legality classification, demonstrating the complementary value of these modalities.
- Transformer-based video backbones, specifically Video Swin Transformer, generally outperform CNN-based architectures in recognizing complex rider maneuvers and assessing legality in dense and noisy real-world traffic.
- Eye-gaze acts as a crucial attention cue, and telemetry provides essential kinematic information, both of which are indispensable for precise understanding and prediction of rider actions and their compliance with traffic regulations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.