Reproduce “GoiBERT: Compact Vietnamese Language Models via Two-Stage Knowledge Distillation” on OpenResearch
Turn your favorite coding agents into research agents that review literature, develop hypotheses, and run experiments.
Universal app for Apple silicon and Intel. Requires macOS 11 or later.
About this paper
Quang Anh Kiet Tran, Nam Anh Tran, Manh Hung Trinh, Thanh Danh Le, Khang Nhat Hoang Vo, Huynh Dai Nhan Tran
GoiBERT uses two teachers to build compact Vietnamese encoders that accept raw text without external word segmentation. A same-tokenizer Vietnamese teacher transfers representations, while a differently tokenized teacher supplies task-level predictions. GoiBERT-large averages 81.13 across eight tasks versus BamiBERT’s 80.65, with 49.4M rather than 102.4M parameters and about twice the encoder throughput under tested conditions; results are weaker on NLI and NER.
Summary by alphaXiv.
Tip: change arxiv.org to autoarxiv.org in any arXiv abstract link to open the paper here.