Decision Trees & Splitting

A decision tree partitions data by asking yes/no questions. At depth 1 it makes a single axis-aligned cut — each colored region predicts one class. One cut isn't enough for interleaved clusters.

Strong split — the line lands between clusters. The left side is mostly red (100% pure) and the right mostly green (100% pure). Low weighted Gini (0.00) confirms the children are much purer than the parent.

With only 1 split, the tree picks the single best feature and threshold — but one cut can't separate interleaved clusters.

Underfitting — train 79% and test 92% are both low. The tree is too shallow to capture the cluster structure.

No pruning — this is the same depth-8 overfit tree from step 4 with 18 leaves. Train 92% but test only 77%. Increase alpha to start removing noise-fitting branches.

More depth creates finer axis-aligned rectangles. Unlike SVMs or neural networks, trees cannot create diagonal or curved boundaries.