Reproduce “On the Use of Deep Learning Models for Semantic Clone Detection” on OpenResearch

About this paper

Subroto Nag Pinku, Debajyoti Mondal, Chanchal K. Roy

A multi-faceted evaluation of deep learning models for semantic code clone detection reveals that cross-language models, particularly C4 from the University of Saskatchewan, exhibit superior robustness and semantic understanding across diverse benchmarks, including LLM-generated data, compared to single-language models. The work provides a comprehensive comparison, highlighting the importance of appropriate datasets and demonstrating C4's consistent top performance and resilience to code mutations.

Key insights

• Cross-language deep learning models, particularly C4, demonstrate superior or competitive performance for single-language semantic clone detection due to their ability to learn robust, language-agnostic code representations.
• Models fine-tuned on specific semantic clone datasets (e.g., GPTCloneBench) show a much stronger ability to detect semantic clones compared to those trained solely on syntactic clone datasets like BigCloneBench.
• Transformer-based models (CodeBERT, C4) exhibit high robustness to common code mutations, with C4 showing minimal performance degradation (less than 1% F1 change) across various mutation types.

Summary by alphaXiv.