ENID
Lensa ML

Data Augmentation

Step 1 of 6: Why Augment?

Small datasets overfit — augmentation multiplies your data

Training Set Size50
EpochLoss0369TrainValVal (aug)illustrative
Dataset size
50
Overfit gap
6.49
Augmented views
250
Aug gap
1.79

With so few samples, the model memorizes the training data and validation loss pulls away from training loss. Augmentation (orange line) narrows the gap by adding varied copies.

Drag Training Set Size from 10 to 1,000. The curves are a schematic model, and the demo assumes augmentation makes the dataset 5 times larger. Augmented copies are not independent samples, though: a flipped cat is still the same cat, so the real gain is smaller than the multiplier suggests. Augmentation works by building in knowledge about which changes should not alter the label. AlexNet (2012) trained on random crops and horizontal flips, and its authors reported substantial overfitting without them.

Step 2 of 6: Geometric Transforms

Rotate, flip, crop — spatial invariances

Rotation Angle (degrees)0
OriginalTransformed0°Common geometric transforms:Rotate Flip Crop Shear Scale TranslateEach creates a valid new training sample from existing data
Angle
0°
Direction
None
|Rotation|
0°

Small rotations (< 30°) are the safest geometric augmentation. The object keeps its natural orientation, and the model still sees new variation.

Drag Rotation Angle from −180° to 180°. A transform is safe only if it keeps the label: a horizontal flip suits cats and cars but not text or road signs with arrows. For detection and segmentation, the same geometric transform must also be applied to the boxes or masks, or the labels stop matching the image. Random resized cropping, which samples a region of the image and rescales it, is the standard ImageNet augmentation. Rotations by angles other than multiples of 90° also leave empty corners that need padding or cropping.

Step 3 of 6: Color & Intensity

Brightness, contrast, and color jitter

Brightness Shift0.00
Simulated image pixelsPixel intensity distribution0265177102128154179205230No changeAlso: contrast, saturation, hue jitter, color dropout
Brightness
0.00
Avg original
127
Avg adjusted
127

Subtle shifts simulate natural lighting variation. The histogram barely moves, so pixel statistics stay close to the original.

Drag Brightness Shift from −1 to 1 and compare the Avg Original and Avg Adjusted cards. Brightness is one part of color jitter, which also randomizes contrast, saturation and hue. These photometric transforms change pixel values but not geometry, so boxes and masks need no update. Self-supervised learning relies on them heavily: the SimCLR paper (Chen et al., 2020) found random cropping combined with color distortion to be the most important pair of augmentations. Medical images often call for intensity changes that mimic real scanner variation.

Step 4 of 6: Cutout & Mixup

Occlusion and interpolation strategies

Cutout side (% of image side)20
Mixup weight λ (share of image B)0.30
CutoutMasks out 4% of the areaMixup (λ=0.30)label = 70% A + 30% BCutout: forces model to use non-local featuresMixup: smooths decision boundaries between classesCutMix combines both: paste a patch from another image
Cutout %
20%
Mixup λ
0.30
Visible %
96%

Moderate cutout (10-30% of the side) makes the model use more of the image instead of one telltale region. Mixup blends the pixels and the labels with the same weight λ, so the target becomes a soft label, which tends to improve calibration. In practice λ is drawn from Beta(α, α) for each pair; α is the hyperparameter.

Drag the Cutout side slider from 0 to 50 and the Mixup weight λ slider from 0 to 1; the Mixup λ card shows the blend weight. Cutout (DeVries & Taylor, 2017) blanks a random square. Mixup (Zhang et al., 2017) blends two images and their labels with the same weight λ: x = λx₁ + (1 − λ)x₂ and y = λy₁ + (1 − λ)y₂, with λ drawn from a Beta(α, α) distribution. CutMix (Yun et al., 2019) pastes a patch from one image onto another and mixes the labels by patch area.

Step 5 of 6: Augmentation Probability

How often to apply each transform

Augmentation Probability0.50
Training batch — augmented samples highlightedaugorigorigaugorigaugorigaugorigaugaugorig
Probability
0.50
Augmented
6/12
Expected (p × 12)
6.0

A moderate probability (0.2-0.7) mixes originals and augmented samples. Each epoch sees different variations, which raises the effective diversity of the training set.

Drag Augmentation Probability from 0 to 1 and watch which of the 12 copies are marked AUG. Each transform in a pipeline usually has its own probability, and augmentation runs on the fly, so every epoch sees a fresh random version of each image rather than a fixed enlarged dataset. The Expected card is p × 12, the average number of copies marked AUG. Tuning these settings by hand is slow. AutoAugment learned a policy with reinforcement learning, and RandAugment cut the search to two numbers: how many transforms to apply and how strong they are.

Step 6 of 6: When Augmentation Hurts

Too much augmentation degrades performance

Augmentation Strength0.30
Augmentation Strength (schematic curve)Val Accuracy0.950.50sweet spotno augunderover-aug
Strength
0.30
Val accuracy
0.920
vs baseline
+10.0 pts
Regime
Sweet spot

Enough augmentation to teach useful invariances without distorting the data distribution. Validation accuracy peaks here, well above the no-augmentation baseline.

Drag Augmentation Strength from 0 to 1. The accuracy curve is schematic: it starts at the no-augmentation baseline of 0.82, peaks at 0.92 at a strength of 0.3, and drops below the baseline past 0.7. The best strength depends on the model and data. Large models trained for many epochs usually tolerate, and need, stronger augmentation, while small models can underfit with too much. Augmentation also hurts when it breaks labels, such as flipping digits or mirror-image letters. Test-time augmentation reuses the transforms at inference, averaging predictions over several augmented copies of each input.

What is data augmentation?

Data augmentation is a training technique that creates new examples by applying label-preserving changes to existing ones, such as flipping, cropping or brightening an image. The model sees many variations of each example, which makes it less likely to memorize the training set and better at handling the variation it will meet in real data.

Augmentation is standard in computer vision, where nearly every image classifier, detector and segmentation model trains with random crops, flips and color jitter. It is also used for audio, with time shifts, added noise and SpecAugment's masking of spectrogram bands, and for text, with back-translation and synonym replacement. Self-supervised methods such as SimCLR learn by comparing two augmented views of the same image.

Key ideas

  • Augmentation creates new training examples by transforming existing ones without changing their labels.
  • It acts as a regularizer and reduces overfitting, especially on small datasets.
  • Geometric transforms move pixels around, and photometric transforms change their values.
  • For detection and segmentation, geometric transforms must also be applied to the boxes or masks.
  • Mixup and CutMix blend both the images and their labels.
  • Too much augmentation, or a transform that changes the label, lowers accuracy.

Common questions

Should you augment the validation and test sets?

No. Random augmentation belongs only in training, and validation and test images should stay as close as possible to real inputs so the scores mean something. The one exception is deterministic preprocessing, such as resizing and center-cropping, which is applied the same way to every split.

Which image augmentations should you start with?

Random resized crops and horizontal flips are the usual first choice, followed by mild color jitter. For larger models and longer training, add RandAugment, mixup or CutMix. Check each transform against the task, since flips and large rotations can change the meaning of text, digits or medical images.

Does data augmentation work for tabular data?

It is less common, because there is no obvious label-preserving transform for a row of mixed features. Adding small noise to numeric features is one option. For class imbalance, SMOTE creates synthetic minority-class rows by interpolating between neighboring examples.

What is the difference between data augmentation and synthetic data?

Augmentation transforms real examples, so every new sample stays close to one that exists. Synthetic data is generated from scratch, by a simulator, a 3D renderer or a generative model such as a diffusion model. Synthetic data can cover cases the real dataset lacks, but it risks a gap between the generated and real distributions.

Related explainers

References

  1. Shorten & Khoshgoftaar (2019), "A survey on Image Data Augmentation for Deep Learning", Journal of Big Data
  2. Zhang et al. (2017), "mixup: Beyond Empirical Risk Minimization"
  3. DeVries & Taylor (2017), "Improved Regularization of Convolutional Neural Networks with Cutout"
  4. Yun et al. (2019), "CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features"
  5. Cubuk et al. (2019), "RandAugment: Practical automated data augmentation with a reduced search space"

Last updated