SOURCE-LINKED INTELLIGENCE

Fine-Tuning Fixes Mode Collapse and Over-Dispersion in LLMs

arXiv · AI, language, vision and robotics · article · Sep 15, 2026 · UTC

Recent work by Doshi and Hauser (2024), Bisbee et al. (2024), and Xie et al. (2026) raises concerns that outputs from large language models (LLMs) tend to be under-diverse: they repeat or resemble one another more often than responses from the population they are meant to represent, a phenomenon known as mode collapse. In this work, we show that whether mode-collapse, or its opposite, occurs depends on the specific model and dataset used. Further, with sufficient supervised fine-tuning (SFT) data, LLM output diversity converges toward that of the target distribution from which fine-tuning data

Read original source ↗ Open in workspace

recordType

paper

region

Global