Bagging & Random Forests

This sample uses most of the original data. On average, bootstrap sampling includes ~63.2% of unique points — the rest are duplicates drawn with replacement.

Bagging trains 5 trees on different bootstrap samples and combines their votes. The ensemble is more stable than any single tree.

Using 3 of 5 features balances individual tree quality with diversity. This is the sweet spot for Random Forests.

Averaging 5 trees reduces variance substantially. Each additional tree helps smooth the prediction.

With 10 trees, most points have several OOB predictions, giving a reliable error estimate without a separate test set.

With 10 trees, importance estimates are noisy. The ranking may shift between runs.