Reproduce “Benchmarking Large Language Models for Polymer Property Predictions” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Sonakshi Gupta, Akhlak Mahmood, Shivank Shukla, Rampi Ramprasad

Researchers at Georgia Institute of Technology benchmarked fine-tuned general-purpose large language models against established domain-specific machine learning methods for predicting polymer thermal properties, finding that LLMs generally underperformed traditional methods but open-source LLaMA-3 achieved competitive accuracy with domain-specific chemical language models like polyBERT.

Key insights

  • General-purpose LLMs, despite their simplicity, currently do not surpass the predictive accuracy of the best domain-specific traditional machine learning models for polymer properties.
  • Single-task learning is more effective for LLMs in this domain than multi-task learning, suggesting a challenge in leveraging shared representations across different polymer properties.
  • LLMs exhibited catastrophic forgetting during multi-task sequential learning, indicating difficulty in retaining knowledge when iteratively fine-tuned on new, distinct property tasks.

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.