CBOW

Window ±2: CBOW averages 4 context word embeddings into a single vector, then predicts the center word "sat". This is the "bag" in CBOW — context word order doesn't matter.

The embedding for "the" is a 6-dimensional vector. Each context word contributes equally to the average — this is the "bag" assumption. Slide to inspect each context word's embedding.

6 dimensions: the averaged context vector is multiplied by the 12×6 output weight matrix to produce one logit per vocabulary word. The target "sat" has logit -0.06 — next, softmax converts these raw scores into probabilities.

Temperature 1.0: moderate confidence. The target "sat" has probability 0.081. Softmax converts raw logits into a valid probability distribution that sums to 1.

Before training: embeddings are random, prediction probability is ~8% (uniform over 12 words). The loss (cross-entropy) is high because the model can't predict the center word.

CBOW predicts the center word from context — it averages all context embeddings into one vector, producing one update per window. This makes it fast and good for frequent words, but rare words get averaged out since they rarely appear as the center target.