ML//inference//distributional shift//basin of attraction

Basins of attraction are regions of latent space where certain types of continuation are strongly preferred, and the model falls into them depending on the context.

Basins of attraction are regions of latent space where certain types of continuation are strongly preferred, and the model falls into them depending on the context.

Pretraining created these basins: reasoning followed by a coherent conclusion is one basin, a question followed by a direct answer is another, and nonsense followed by an unpredictable continuation is a third.

Random text as context doesn't help because it positions the model in a basin with no semantic structure. The model has no signal about what type of continuation to produce.

Structured reasoning activates a specific basin because during pretraining, that pattern co-occurred systematically with correct conclusions. It's intentional distributional shift toward a useful attractor.

The technical effect is that reasoning tokens reduce the entropy of the output distribution. Fewer plausible tokens survive as continuations, and the ones that remain tend to be more correct.

Overthinking happens when excessive reasoning tokens push the model past the useful basin into a complex reasoning attractor that's counterproductive for simple problems.

Exposure bias is falling into a bad attractor: once errors accumulate, the context vector enters a basin the model never navigated during training.