Reproduce “LaFiCMIL: Rethinking Large File Classification from the Perspective of Correlated Multiple Instance Learning” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Tiezhu Sun, Weiguo Pian, Nadia Daoudi, Kevin Allix, Tegawendé F. Bissyandé, Jacques Klein
LaFiCMIL introduces a method for classifying large files by re-framing the task as Correlated Multiple Instance Learning, treating documents as 'bags' of semantically related 'instances' or chunks. This approach efficiently processes up to 20,000 tokens on a single 32GB GPU, achieving state-of-the-art results on multiple NLP and code defect detection benchmarks while demonstrating significantly reduced computational resource requirements.
Key insights
- Treating large files as 'bags' of correlated 'instances' (chunks) within a c-MIL framework allows for leveraging semantic dependencies to enhance classification accuracy.
- Efficiently approximating the self-attention mechanism within LaFiAttention enables the processing of extremely long sequences (up to 20,000 tokens) on limited hardware (a single 32GB GPU).
- Integrating a learnable category vector helps to iteratively aggregate high-level, correlation-aware features from all chunks into a compact representation for the entire document.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.