ML//multimodal//SAM

The Segment Anything Model (Meta, 2023) is a foundation model for image segmentation: prompted with points, boxes or text, it returns pixel-precise masks for any object. Its image encoder is a vision transformer, the foundation model pattern applied to segmentation.

The Segment Anything Model (Meta, 2023) is a foundation model for image segmentation: prompted with points, boxes or text, it returns pixel-precise masks for any object. Its image encoder is a vision transformer, the foundation model pattern applied to segmentation.

SAM 2 extends it to video, tracking and segmenting objects across frames. Combined with GroundingDINO, which detects objects from text, SAM adds the pixel segmentation for open-vocabulary understanding.