Reproduce “Fine-Grained Behavior Simulation with Role-Playing Large Language Model on Social Media” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Kun Li, Chenwei Dai, Wei Zhou, Songlin Hu
Researchers from the University of Chinese Academy of Sciences developed FineRob, a new dataset for simulating fine-grained social media user behavior, and introduced OM-CoT fine-tuning. This method significantly enhances LLMs' ability to predict specific user actions by explicitly integrating observation and memory-based reasoning, improving performance on tasks by up to 9.8% compared to non-fine-tuned models.
Key insights
- Commercial closed-source LLMs (e.g., GPT-4) consistently outperform open-source models in zero-shot fine-grained behavior simulation, indicating superior capabilities in complex role-playing.
- Optimal performance for behavior simulation is achieved with approximately 30 recent historical behavior entries; exceeding this can introduce noise and decrease accuracy.
- The special tokens (<ANA> for observation, <MEM> for memory) within the OM-CoT framework are crucial, with their impact varying across different fine-grained behavior elements.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.