← Dictionary / Next Sentence Prediction (NSP)

Next Sentence Prediction (NSP)

Appears in 1 paper

BERT's second pre-training objective.

As used in Paper 11 — BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding →

BERT's second pre-training objective. Given two sentences A and B, classify whether B is the genuine next sentence after A or a randomly sampled sentence. Uses the [CLS] token's hidden state. Later research showed this task contributes little and may be removed (RoBERTa).

Appears in papers

Paper 11 — BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding →