ML//RAG//sentence transformer

A sentence transformer is a BERT or RoBERTa model fine-tuned with contrastive learning specifically to produce semantically meaningful sentence embeddings.

A sentence transformer is a BERT or RoBERTa model fine-tuned with contrastive learning specifically to produce semantically meaningful sentence embeddings.

Base BERT produces embeddings optimized for MLM rather than for similarity search, and sentence transformers fix this. The fine-tuning objective gives similar sentences a high cosine similarity and dissimilar ones a low one. OpenAI's embedding models (text-embedding-ada-002 and others) are essentially this: encoder models fine-tuned for similarity.

They are the backbone of modern RAG pipelines.

1Encode the query2Encode the documents3Find the nearest neighboursin a vector database