Reproduce “LiLT: A Simple yet Effective Language-Independent Layout Transformer for Structured Document Understanding” on OpenResearch

About this paper

Jiapeng Wang, Lianwen Jin, Kai Ding

LiLT, a Language-independent Layout Transformer, disentangles document layout understanding from language content by learning layout structures from monolingual data and combining them with off-the-shelf textual models. This approach achieves competitive or superior performance across various structured document understanding tasks and 8 languages, outperforming prior models that relied on significantly larger and more diverse multilingual pre-training datasets.

Key insights

• The visual layout structure of documents is largely language-independent, enabling the learning of generalizable layout knowledge from monolingual data.
• Decoupling layout and text processing permits efficient multilingual adaptation, as a pre-trained language-independent layout component can be combined with any existing monolingual or multilingual textual model.
• A controlled cross-modal interaction mechanism (BiACM with a `DETACH` operation during pre-training) is crucial for maintaining the integrity and reusability of pre-trained text models while effectively integrating layout context.

Summary by alphaXiv.