Reproduce “Probing Knowledge Holes in Unlearned LLMs” on OpenResearch

Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.

Universal app for Apple silicon and Intel. Requires macOS 11 or later.

About this paper

Myeongseob Ko, Hoang Anh Just, Charles Fleming, Ming Jin, Ruoxi Jia

Researchers introduced a dynamic evaluation framework to uncover "knowledge holes" in unlearned large language models, demonstrating that current unlearning methods inadvertently cause substantial loss of benign knowledge in areas both adjacent and semantically unrelated to the forgotten content. The study found standard benchmarks often miss this degradation, with unlearned models frequently generating nonsensical responses to probes, and attempts to mitigate these holes often reintroduce harmful content or create new ones.

Key insights

  • Machine unlearning, while effective at removing harmful content, consistently creates "knowledge holes" leading to significant, unintended loss of benign knowledge in LLMs.
  • Standard, static benchmarks are insufficient for comprehensively evaluating unlearning, as they consistently fail to capture these widespread knowledge degradations.
  • A fundamental trade-off exists in unlearning: mitigating identified knowledge holes often leads to the re-emergence of harmful content or the creation of new knowledge holes, exhibiting an "onion effect."

Summary by alphaXiv.

Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.