Reproduce “The Dobby Report” on OpenResearch

About this paper

Matthias Gallé

Sentient Research and academic collaborators develop the Dobby model family, LLMs designed to exhibit both an "unhinged" persona (soft skills) and maintain technical competence (hard skills), overcoming the typical "seesaw effect" where personality fine-tuning degrades core abilities through novel data overloading techniques that rewrite instruction datasets to incorporate desired tone while preserving accuracy, token-level routing for inference-time skill composition, and an automated "Dobby Factory" pipeline using LLM-guided hyperparameter optimization that converges faster than Bayesian methods, with the Dobby-8B-Unhinged model receiving 53.4% preference votes from 155,000 users in arena testing while maintaining competitive performance on standard benchmarks including MT-Bench scores of 6.39 and restored markdown formatting capabilities.

Key insights

• Data overloading significantly improves the Pareto frontier between instruction following accuracy and persona consistency compared to conventional data mixing approaches
• Token-level routing can maintain performance while reducing computational costs by selectively invoking larger models only for knowledge-critical tokens
• LLM-guided hyperparameter optimization achieves faster convergence and better rewards than traditional Bayesian optimization methods
• Community preference validation through large-scale voting reveals user appetite for distinctive AI personas beyond conventional alignment targets

Summary by alphaXiv.