Reproduce “Hierarchical Process Reward Models are Symbolic Vision Learners” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Shan Zhang, Aotian Chen, Kai Zou, Jindong Gu, Yuan Xue, Anton van den Hengel

This research from Adelaide AIML and Ohio State University presents a self-supervised symbolic auto-encoder that translates diagrams into interpretable, hierarchical logic forms of geometric primitives. The system enhances diagram reconstruction, perception, and mathematical reasoning by generating fine-grained, rule-based rewards for multi-level parsing and stabilizing reinforcement learning with novel techniques.

Key insights

  • Representing diagrams as hierarchical symbolic logic forms, rather than continuous pixel-level features, enables more interpretable, structured, and accurate understanding for diagrammatic content.
  • Hierarchical Process Rewards, which condition higher-level rewards on the correctness of lower-level primitives, are essential for enforcing step-level parsing consistencies and improving understanding of complex relations.
  • Stabilization techniques, such as hard negative contrastive learning and power normalization annealing, effectively mitigate the training instability of reinforcement learning in domains with sparse rewards like diagram reconstruction.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.