Reproduce “ERASMO: Leveraging Large Language Models for Enhanced Clustering Segmentation” on OpenResearch

About this paper

Fillipe dos Santos Silva, Gabriel Kenzo Kakimoto, Julio Cesar dos Reis, Marcelo S. Reis

The ERASMO framework leverages large language models (LLMs) to create high-quality embeddings from tabular data for enhanced clustering segmentation. It consistently achieves superior clustering outcomes, marked by higher Silhouette Scores and Calinski-Harabasz Indices and lower Davies-Bouldin Indices, across diverse real-world datasets compared to existing baseline methods.

Key insights

• Fine-tuning transformer-based language models directly on textually encoded tabular data enables the generation of contextually rich and structurally representative embeddings for clustering.
• Randomly shuffling feature sequences during textual encoding helps the LLM learn robust representations that are independent of arbitrary feature ordering in tabular data.
• Verbalizing numerical tokens can further improve an LLM's understanding of numerical context, leading to more accurate embeddings and enhanced clustering performance in certain datasets.

Summary by alphaXiv.