Reproduce “Sensi: Learn One Thing at a Time -- Curriculum-Based Test-Time Learning for LLM Game Agents” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mohsen Arjmandi
The Sensi LLM agent architecture significantly improves sample efficiency for test-time learning in unknown game environments by employing a curriculum-based approach and external knowledge management. Sensi v2 completed its entire learning curriculum in roughly 32 action attempts, representing a 50–94x reduction in interactions compared to baselines, though perceptual hallucinations prevented it from solving game levels.
Key insights
- Decoupling cognitive functions into distinct LLM roles, combined with external symbolic control, allows for a more structured and sample-efficient learning process in novel environments.
- The use of an internal 'sense_score' generated by an 'LLM-as-judge' can act as an effective intrinsic motivation and control signal for guiding curriculum progression and knowledge validation.
- Managing the agent's full cognitive state in an external database overcomes LLM context window limitations and enables robust, steerable learning trajectories across numerous interactions.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.