ML//diffusion model//DiT

A DiT (Diffusion Transformer) is a diffusion model architecture that uses a standard transformer as its backbone in place of the U-Net, and it is what large image and video generators are built on today. The image is patched into tokens, transformer blocks are applied, and the tokens are unpatched to reconstruct the image.

A DiT (Diffusion Transformer) is a diffusion model architecture that uses a standard transformer as its backbone in place of the U-Net, and it is what large image and video generators are built on today. The image is patched into tokens, transformer blocks are applied, and the tokens are unpatched to reconstruct the image.

Sora, Flux and SD3 all use DiT variants, because transformers scale better than U-Nets at large model sizes. The architectures have converged: image generation now uses the same blocks as language models.