Reproduce “Length Representations in Large Language Models” on OpenResearch

About this paper

Sangjun Moon, Dasom Choi, Jingun Kwon, Hidetaka Kamigaito, Manabu Okumura

This research empirically investigates how output sequence length is encoded within Large Language Models (LLMs), revealing that length information is predominantly localized in specific multi-head attention outputs of early transformer layers. The study demonstrates that scaling these identified hidden units can control output length across various generative tasks while largely preserving semantic informativeness, suggesting a partial disentanglement of length from content representations.

Key insights

• Length information is consistently and most strongly encoded in the multi-head attention outputs, particularly within the second transformer layer, across various LLMs and prompting methods.
• Length-specific prompts activate distinct sets of individual hidden units responsible for encoding length, and these units become more dominant in conveying length information.
• Length information is partially disentangled from semantic content, allowing for direct manipulation of output length by scaling specific hidden units without significantly compromising generated text informativeness.

Summary by alphaXiv.