Image Segmentation
Segmentation vs Detection
From image-level labels to pixel-level masks
Classification assigns one label to the entire image. Quick, but no spatial information — we only know WHAT is in the scene, not WHERE.
Step 1 of 6: Segmentation vs Detection
From image-level labels to pixel-level masks
Classification assigns one label to the entire image. Quick, but no spatial information — we only know WHAT is in the scene, not WHERE.
Step 2 of 6: Semantic Segmentation
Every pixel gets a class label — sky, road, car, person
With 3 classes, the model must learn distinct boundaries between each region. Per-class counts: Sky=46, Tree=17, Road=37.
Step 3 of 6: Encoder-Decoder Architecture
Compress then expand — the U-shaped backbone of segmentation
Depth 2: one downsampling step halves the spatial resolution while doubling channels. The decoder upsamples back to the original size.
Step 4 of 6: Skip Connections
U-Net's key insight: bridge encoder details to the decoder
Skip connections (U-Net style) concatenate encoder features with decoder features at matching resolutions. This preserves fine spatial details — boundaries are sharp and IoU jumps from 0.64 to 0.87.
Step 5 of 6: Instance Segmentation
Same class, different objects — each gets its own mask
2 objects of the same class. Semantic segmentation colors both identically. Instance segmentation assigns each a unique ID and mask — critical for counting and tracking.
Step 6 of 6: Upsampling Methods
Nearest, bilinear, or learned — how to grow feature maps back
Nearest neighbor: each output pixel copies the closest input pixel. Fast and parameter-free, but produces blocky artifacts — visible as 2×2 blocks of identical values.