How Dimensionality Reduction Works

As dimensions increase, the same number of data points become exponentially sparser. In high dimensions, most of the space is empty — data gets lonely. This is the curse of dimensionality.

PCA finds the axis that captures the most spread (variance) in the data. Rotate the line to maximize the explained variance — that is the first principal component.

The scree plot shows variance captured by each component. Look for the elbow — the point where adding more components gives diminishing returns. Fewer components means more compression but higher reconstruction error.

PCA compresses 5 features into 2 components, then reconstructs the original. More components = lower error, but less compression. The red shows information lost.

t-SNE maps high-dimensional data to 2D while preserving local neighborhoods. Similar points stay close. The perplexity parameter controls how many neighbors each point considers — higher values reveal broader structure.

Use PCA when you need fast, invertible dimensionality reduction (preprocessing, denoising). Use t-SNE when you want to visualize clusters — but remember, distances between clusters are not meaningful.