Reproduce “Learning to Compose Task-Specific Tree Structures” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Jihun Choi, Kang Min Yoo, Sang-goo Lee
Gumbel Tree-LSTM learns a binary, task-specific tree directly from unparsed text, using discrete composition choices in the forward pass and a differentiable Straight-Through Gumbel-Softmax estimator to train those choices. It achieves competitive results on natural language inference and sentiment benchmarks, including 85.6% accuracy on SNLI and 90.7% on SST-2, while using fewer parameters than some comparison systems.
Key insights
- The learned hierarchy can vary with the downstream task: models trained on different objectives sometimes group the same sentence differently, without receiving parse-tree supervision.
- Contextualizing leaf representations with an LSTM improved the reported SNLI results over the corresponding model without that component.
- The approach makes discrete latent tree construction compatible with gradient-based training, although its straight-through gradient estimator is biased.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.