What is text classification?
Text classification is the task of assigning a label to a piece of text, such as positive or negative for a review, or spam or not spam for an email. A classifier turns the text into a vector of numbers, computes a score for each label, and picks the label with the highest score. It learns how to score from examples that people have already labeled.
It is one of the most common uses of machine learning on language. Email providers filter spam with it, support teams route tickets by topic, chatbots detect user intent, platforms flag toxic posts, and companies track sentiment in reviews and social media. The methods range from word counts with logistic regression, which still make strong baselines, to fine-tuned transformers such as BERT and zero-shot prompting of large language models.
Key ideas
- A text classifier maps a document to a vector and the vector to a label.
- Bag-of-words and TF-IDF vectors with a linear model make a strong, fast baseline.
- A linear classifier separates classes with a hyperplane w · x + b = 0.
- Softmax turns class scores into probabilities, and temperature controls how confident they look.
- Lexicon scoring needs no training but fails on negation, sarcasm, and context.
- Multi-class tasks use one softmax over K classes; multi-label tasks use one sigmoid per label.
Common questions
What is the best algorithm for text classification?
It depends on how much labeled data there is. With a few thousand examples, TF-IDF with logistic regression or a linear SVM is fast and hard to beat. With more data, or when context and negation matter, a fine-tuned transformer such as BERT is usually more accurate. With no labeled data at all, a large language model can classify from a prompt.
How do you evaluate a text classifier?
Accuracy works when classes are balanced. When one class is rare, such as spam or toxic posts, precision, recall, and F1 for that class are more informative, because a model that always predicts the majority class can still score high accuracy. A confusion matrix shows which classes get mixed up.
How much labeled data does text classification need?
Simple binary tasks with distinctive words can work with a few hundred labeled examples. Fine-tuning a pretrained model needs far less data than training from scratch, because the model already knows the language. Tasks with many classes or subtle distinctions need more examples per class.
What is the difference between sentiment analysis and text classification?
Sentiment analysis is one kind of text classification, where the labels are opinions such as positive, negative, and neutral. Text classification also covers topic labeling, spam detection, language identification, and intent detection, all of which use the same methods with different labels.
Related explainers
- TF-IDF — Score how important a word is to a document in a collection — the backbone of search engines and text retrieval
- Bag of Words — Turn text into numbers — count word occurrences to create the simplest document representation in NLP
- Naive Bayes Classifier — Probabilistic classification with Bayes' rule
- Softmax, Logits & Temperature — Turning raw scores into probabilities
- BERT & Masked Language Modeling — Mask a word, predict it from both sides — how BERT learns deep bidirectional representations
- Model Evaluation & Metrics — Precision, recall, F1, ROC curves, and regression metrics
References
- Pang, Lee & Vaithyanathan (2002), "Thumbs up? Sentiment Classification using Machine Learning Techniques", EMNLP
- Joulin et al. (2016), "Bag of Tricks for Efficient Text Classification"
- Guo et al. (2017), "On Calibration of Modern Neural Networks", ICML
- Devlin et al. (2019), "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding", NAACL
- Jurafsky & Martin, "Speech and Language Processing" (3rd ed. draft)
Last updated