Reproduce “Kolibri: A Sovereign European Model on the Pareto Frontier” on OpenResearch

About this paper

Aleph Alpha

Kolibri is an open-weight bilingual model designed for local deployment on controlled infrastructure. It activates 3.46B of 78.1B parameters per token and uses sliding-window attention in 40 of 50 blocks, with full attention in every fifth block. Against Kolibri Origin, its post-trained scores are higher in English and German, with 2.7× reported decode throughput under the paper’s measurement setup.

Key insights

• Kolibri combines sparse computation with substantial model capacity, aiming to balance quality and inference cost rather than maximize performance through active parameter count alone.
• German capability is addressed throughout the pipeline—from data curation and tokenization to German reasoning traces—instead of treating German as a translation-only target.
• The reported quality–throughput position depends on the chosen benchmarks and serving setup; internal evaluation sets, acknowledged contamination, and uneven long-context results limit broader conclusions.

Summary by alphaXiv.