Feature Extraction

Every ML model starts with a table of data. Each row is one example (a house). Each column is a feature — a measurable property. The last column is the target — what we want to predict. Feature extraction is about making these columns as useful as possible.

A feature is a single measurable property — one column in your data. Numerical features like square footage have magnitude and spread. Categorical features like "pool" encode discrete groups. Understanding each feature's distribution is the first step to deciding how to transform it.

Price per Sq Ft: Normalizes price by size — a 3000 sqft house costing $600k is cheaper per foot than a 1000 sqft house at $300k. Feature engineering is the art of creating new columns from existing ones that make patterns easier for the model to see. Good features often encode domain knowledge — things you know about the problem that raw numbers don't capture.

Feature scaling prevents features with large magnitudes from dominating. Min-Max squeezes values into [0, 1]. Z-Score standardization centers at zero with unit standard deviation. Most algorithms — gradient descent, KNN, SVM — need scaled features to work properly.

PCA finds the axis of maximum variance — the direction along which data spreads the most. It then projects every point onto that line. Here, PC1 captures 80% of the information from 2 features in just 1 number. In real ML, PCA can compress hundreds of features into a handful of principal components.

Feature importance scores rank how much each feature contributes to predictions. Features below the threshold add noise without signal. Dropping them gives a simpler model that generalizes better — fewer features means less overfitting, faster training, and easier interpretation.