Reproduce “CAPTURE: Disentangling Preference Drift from Memory Poisoning in Personalized LLM Agents” on OpenResearch

About this paper

S M Asif Hossain, Ruksat Khan Shayoni, Md Kishor Morol

CAPTURE frames personalized-agent memory updates as an inference problem: an apparent preference change may be either genuine user drift or adversarial memory poisoning. Its temporal authenticity gate, multi-timescale memory ledger, selective clarification, and independent safety constraint achieve a 71.5% personalization win rate, 11.5% poisoning success, and 83.5% adherence to genuine updates on the D-PREFGUARD benchmark, though the security advantage narrows against adaptive attackers.

Key insights

• Preference authenticity is a latent-state estimation problem rather than a property that can be determined from source or recency alone; full interaction history can resolve cases that those features cannot.
• Representing scope and timescale helps distinguish a temporary or domain-specific request from a permanent revision, with CAPTURE choosing “scope” in 71% of context-switch cases versus 4% for StateMem.
• Clarifying questions are useful when their cost and error risk are lower than the uncertainty of making an immediate memory decision, while safety thresholds should remain outside the preference-learning loop.

Summary by alphaXiv.