Reproduce “JsDeObsBench: Measuring and Benchmarking LLMs for JavaScript Deobfuscation” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Guoqiang Chen, Xin Jin, Zhiqiang Lin

JsDeObsBench introduces the first comprehensive benchmark for evaluating Large Language Models (LLMs) in JavaScript deobfuscation, providing a new dataset of 36,260 obfuscated benign and 4,515 malicious JS programs with ground truth. The study quantifies LLMs' effectiveness across various obfuscation techniques and evaluation metrics, demonstrating their superiority in code simplification and readability while identifying challenges in semantic and syntactic correctness, especially for malicious code.

Key insights

  • LLMs excel at simplifying and improving the readability of obfuscated JavaScript, outperforming traditional tools in generating more human-understandable code.
  • Maintaining perfect syntactic and semantic correctness is a significant challenge for current LLMs, particularly when dealing with multiple or complex obfuscation techniques and malicious code.
  • Code-specific LLMs generally demonstrate better performance than general-purpose LLMs in deobfuscation, suggesting the benefit of domain-adapted training.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.