Reproduce “Same Context, Different Agents: A Behavioral Study of Semantic Layers, Multi-Agent Orchestration and Memory Compaction in Data Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Zhivar Sourati, Marianne Menglin Liu, Zheng Wang, Rongguang Wang, Sai Ashish Somayajula, Tao Sheng, Kyu J. Han, Sujith Ravi and 1 more
The paper tests how three context-engineering techniques affect data agents with different tool-use habits. Across seven models, 1,514 tasks, and five benchmarks, it compares semantic layers, orchestration, and compaction. Orchestration raises premature answerers’ accuracy by 22.8 percentage points at four times baseline cost, while interventions can hurt investigators; choosing the wrong data after inspection remains unresolved.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.