Semantic and Instance Segmentation

Wed, 11 Nov 2026 21:00:00 +1100

A classifier says there is a bolt in this picture. A detector says here are fifty boxes, and these are the bolts. Dense prediction goes all the way down: it gives a label to every pixel. That is what you need to measure a tumour in a CT slice, map the road surface in an aerial photograph, or count cells that press against each other in a micrograph.

This page covers the networks that do it. Fully convolutional encoder–decoders and U-Net handle semantic segmentation, where each pixel gets a class. Mask R-CNN handles instance segmentation, where each object also gets its own mask. Along the way it covers the upsampling layers every decoder needs, including one output-size trap that catches nearly everyone. The page assumes convolution, pooling and the training loop from CNN from scratch, and region proposals and RoI pooling from the Object Detection page. Further reading is Szeliski (2nd ed.) §5.4.1, §6.4 and §6.4.2.

Transposed-Convolution on Aayush Bajaj's Augmenting Infrastructure

Semantic and Instance Segmentation