How does a model that predicts brain activity become one that reads it? · Lobeworks/17

An encoding model maps a stimulus to the activity it should produce.

How does a model that predicts brain activity become one that reads it?

An encoding model maps a stimulus to the activity it should produce. Decoding runs it backwards: a generator proposes candidate stimuli (words from a language model, images from an image model), the encoding model predicts the activity each one would cause, and the candidate whose prediction best matches the recording wins. The 2023 fMRI decoder of continuous language worked this way, so its output can only be as faithful as the encoding model is accurate, and anything the recording does not distinguish is filled in by the generator.