ML//multimodal//vision transformer

The vision transformer (ViT) is an image model built on a standard transformer with no convolutions: it chops an image into 16×16 patches, treats each patch as a token and runs the sequence through the transformer.

The vision transformer (ViT) is an image model built on a standard transformer with no convolutions: it chops an image into 16×16 patches, treats each patch as a token and runs the sequence through the transformer.

Given enough data it is surprisingly competitive with CNNs, and it marked the moment transformers started eating computer vision too.