Text Classification
Bag of Words
From text to feature vectors
With a 6-word vocabulary, the sentence becomes a 6-dimensional vector. Most entries are 0 (83% sparse). BoW ignores word order — "great and amazing" = "amazing and great".
Step 1 of 6: Bag of Words
From text to feature vectors
With a 6-word vocabulary, the sentence becomes a 6-dimensional vector. Most entries are 0 (83% sparse). BoW ignores word order — "great and amazing" = "amazing and great".
Step 2 of 6: Feature Space
Documents as data points
This sentence has 2 positive words and no negative words — it maps to the bottom-right of the plot. Notice how all positive reviews land in this region, away from negative ones.
Step 3 of 6: Training a Classifier
Finding a decision boundary
Only 0% accuracy — the boundary is far from optimal. Try rotating to ~135° where positive words (x-axis) and negative words (y-axis) create natural separation.
Step 4 of 6: Softmax & Confidence
From scores to probabilities
Temperature 1.0: the model assigns 97% to Positive and 3% to Negative. At T=1 (standard softmax), the probabilities reflect the raw logit gap.
Step 5 of 6: Sentiment Polarity
Positive, negative, and neutral
Positive words (great, amazing) contribute +0.40, +0.48. These sum to +0.88, giving a clear positive polarity. Lexicon-based scoring is simple but effective for obvious sentiment.
Step 6 of 6: Beyond Binary
Multi-class text classification
Binary classification: just positive vs. negative. One decision boundary separates two regions. This is the simplest case — a single threshold on the sentiment score.