Reproduce “Learning Query-Aware Budget-Tier Routing for Runtime Agent Memory” on OpenResearch

About this paper

Haozhen Zhang, Haodong Yue, Tao Feng, Quanyu Long, Jianzhu Bao, Bowen Jin, Weizhi Zhang, Xiao Li and 3 more

BudgetMem introduces a framework for managing runtime agent memory by employing a query-aware, budget-tiered modular pipeline and a reinforcement learning router to explicitly control performance-cost trade-offs. The framework surpasses existing memory-augmented LLM agents in task performance while offering robust control over computational costs, achieving, for instance, a Judge score of 60.50 on LongMemEval compared to 48.51 for the strongest baseline.

Key insights

• BudgetMem provides explicit and controllable performance-cost frontiers for runtime memory, consistently outperforming previous baselines in both high-performance and budget-constrained settings.
• Implementation and capacity tiering offer a broader range of cost and performance trade-offs, effectively widening budget coverage, while reasoning tiering provides fine-grained quality refinement within a concentrated cost range.
• Reward-scale alignment is crucial for stable reinforcement learning training, preventing optimization from trivial cost minimization and enabling the router to make balanced and effective tier selections.

Summary by alphaXiv.