Reproduce “A.S.E 2.0: Understanding How Repository Context Impacts the Security of LLM-Generated Code” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Jiongchi Yu, Keke Lian, Junqing Wang, Zhikang Fan, Zhaoxuan Li, Yongheng Huang, Ziming Zhao, Dong Zhang

The authors test repository context's effect on secure code generation across 145 CASE-Bench tasks. Across six models, functionality-oriented methods raise functional pass rates by up to 8.2 percentage points, while secure pass rates rise by at most 3.7 points, with no corrected significant differences. Removing safeguards lowers secure pass rates by up to 8.1 points; results are PoC-relative.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.