Reproduce “Multi-label Sequential Sentence Classification via Large Language Model” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Mengfei Lan, Lecheng Zheng, Shufan Ming, Halil Kilicoglu
Researchers from the University of Illinois Urbana-Champaign's ScienceNLP-Lab developed LLM-SSC, a framework that applies Large Language Models to multi-label sequential sentence classification. This framework, enhanced by an auto-weighting contrastive loss (WeighCon), achieved a Micro F1 of 0.907 on their newly introduced BIORC800 dataset, which provides manually annotated multi-label rhetorical roles for biomedical abstracts.
Key insights
- Large Language Models (LLMs) can be effectively applied to sequential sentence classification (SSC), including multi-label scenarios, by leveraging their long context windows and generative capabilities.
- Scientific sentences often inherently serve multiple rhetorical functions, making multi-label classification a more realistic approach than traditional single-label methods.
- Parameter-efficient fine-tuning (PEFT) techniques, such as LoRA, enable the adaptation of large LLMs for specialized tasks like SSC with minimal computational overhead, updating only about 4% of parameters.
- Novel contrastive learning strategies, like auto-weighting (WeighCon), can address challenges in multi-label classification by relaxing positive pair definitions and mitigating class collision issues.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.