Reproduce “CodeDSI: Differentiable Code Search” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Usama Nadeem, Noah Ziems, Shaoen Wu
CodeDSI applies the Differentiable Search Index (DSI) paradigm to neural code search, training a sequence-to-sequence model to directly map natural language queries to code sample identifiers. This approach consistently outperformed state-of-the-art dual-encoder baselines by 2-6% in absolute accuracy across various dataset sizes, achieving up to 73.26% accuracy on a 50K code sample dataset.
Key insights
- The Differentiable Search Index (DSI) paradigm can be effectively adapted for neural code search, offering a unified, end-to-end retrieval mechanism that inherently addresses the semantic gap between natural language queries and code.
- The structure of document identifiers (docids) critically impacts DSI performance, with numerical docids consistently outperforming character-based ones due to how tokenizers process digit sequences.
- Semantically clustered docids, generated by hierarchically grouping code embeddings, provide more robust and often superior performance compared to arbitrary sequential identifiers, especially with character-based representations.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.