Reproduce “Concept than Document: Context Compression via AMR-based Conceptual Entropy” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kaize Shi, Xueyao Sun, Xiaohui Tao, Lin Li, Qika Lin, Guandong Xu
A new unsupervised context compression framework enhances Large Language Model performance by quantifying conceptual importance within Abstract Meaning Representation (AMR) graphs. This method distills essential semantic information from lengthy documents, leading to improved reasoning accuracy and reduced computational overhead for LLMs in knowledge-intensive tasks.
Key insights
- LLMs can effectively reconstruct semantic scenarios and infer implicit relationships from a concise set of core conceptual nodes, even without explicit AMR semantic edges, aligning with cognitive hypotheses.
- Information entropy, when applied to AMR concept nodes, offers an unsupervised and principled method for identifying and preserving essential semantic information crucial for LLM reasoning.
- Integrating structured linguistic representations like AMR with information-theoretic principles provides a powerful framework for context engineering, bridging human language understanding and LLM processing efficiency.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.