Reproduce “Improvement of Human-Object Interaction Action Recognition Using Scene Information and Multi-Task Learning Approach” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Hesham M. Shehata, Mohammad Abdolrahmani

A skeleton-based human-object interaction recognizer augments body-joint graphs with scene nodes representing fixed interaction regions around ATMs and check-in/check-out machines. On a custom four-class dataset from seven cameras, adding these scene features raised reported accuracy from 96.5% to 99.25% at 50 epochs, while GRU temporal modeling and multi-task learning provided less consistent gains.

Key insights

  • Explicit spatial context was more consistently useful than the auxiliary multi-task objective, indicating that knowing where an interaction can occur is central to the task.
  • Temporal GRU layers and the deeper SPGCN architecture did not produce a clear advantage over simpler graph models at the best reported training setting.
  • The approach is tailored to stationary, known objects and interaction zones; its assumptions limit direct transfer to changing environments, moving objects, or unconstrained human-object interactions.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.