Reproduce “Goal Encoding in LLMs” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Arnab Sen Sharma, Eric Todd, Sheridan Feucht, Andy Arditi, Rohit Gandikota, Natalie Shapira, Gabriele Sarti, Constanza Fierro and 2 more
The authors identify sparse attention heads that encode an upcoming coding subgoal and use GoalScope to turn their activations into a description of the next action. Redirecting these heads to another task specification switches the model to that task’s subgoal, including across programming languages. The evidence concerns prompted coding tasks, not robust obstacle persistence or goals in a broader sense.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.