ML//model//BERT//downstream layer

A downstream layer is a simple linear layer on top of BERT's output that converts the embedding vector into task-specific predictions. The name says where it sits: the task-specific part that comes after the general-purpose encoder.

A downstream layer is a simple linear layer on top of BERT's output that converts the embedding vector into task-specific predictions. The name says where it sits: the task-specific part that comes after the general-purpose encoder.

It always follows an encoder, taking the pre-trained representation and adapting it.

Its outputs range from two classes for sentiment analysis to a similarity score for RAG, NER tags per token and entailment classification.

For sentence-level tasks its input is typically the vector of the [CLS] token.