Comp9517 on Aayush Bajaj's Augmenting Infrastructure

Classical Segmentation

Wed, 11 Nov 2026 21:00:00 +1100

Counting cells, measuring a tumour or picking nuts off a conveyor all need one step between “an image” and “numbers about what is in it”: deciding which pixels belong together. That step is segmentation: give every pixel a region label so that pixels with the same label form one meaningful segment. 𐃏

From Convolution to CNNs

Wed, 11 Nov 2026 21:00:00 +1100

Hand-designed features (colour histograms, SIFT, HOG) followed by a trained classifier was the recipe for image recognition until about 2012. A convolutional neural network learns both halves at once, straight from the pixels. This page builds one from the bottom: why the obvious learned classifiers fail on images, what a convolutional layer does instead, the two formulas that size every layer, pooling and receptive fields, what the filters actually learn, and why very deep networks only train once you give them residual connections. 𐃏

Morphology: Counting Nuts and Bolts

Wed, 11 Nov 2026 21:00:00 +1100

A threshold almost never hands you a clean mask. You get specks of noise in the background, pinholes and cracks in the objects, ragged edges, objects cut off by the frame and objects that touch. Mathematical morphology is the toolkit for repairing such masks. It treats a binary image as a set of pixels and probes it with a small set of offsets. Everything is built from two primitives, one that grows objects and one that shrinks them, together with plain set algebra. There are no derivatives and no weights: the operations are nonlinear, and they judge structures by size and shape rather than by contrast.

Motion and Tracking

Wed, 11 Nov 2026 21:00:00 +1100

A single image says what is there. A video also says what moved, and the motion is often the most useful thing in it: it separates a walker from the pavement, tells a car how it is turning, and lets a microscope follow a particle too faint to see in any one frame. This page climbs three rungs. First, where did something change (change detection)? Second, how far and which way did each pixel move (optical flow, with Harris corners explaining where that question even has an answer)? Third, where is the object now, and where will it be next, given a whole history of noisy detections (tracking as Bayesian filtering, solved exactly by the Kalman filter and approximately by the particle filter). 𐃏

Object Detection

Wed, 11 Nov 2026 21:00:00 +1100

A classifier answers “what is in this image?”. A detector answers two questions at once: what objects are present, and where each one is, as a box drawn tightly around it. The output is no longer one label but a list of (class, box) pairs whose length is not known in advance. 𐃏

Seeing in Frequency

Wed, 11 Nov 2026 21:00:00 +1100

Spatial filtering slides a small kernel over the pixels. This page looks at the same image through a different lens: as a sum of waves. In that view blurring keeps the slow waves and throws away the fast ones, a periodic artefact is a handful of bright dots that can be deleted one by one, and an edge detector is a filter that keeps a ring of middle frequencies. The Fourier transform is the change of lens; the convolution theorem is what makes it worth the trouble, because it turns convolution into multiplication. 𐃏

Semantic and Instance Segmentation

Wed, 11 Nov 2026 21:00:00 +1100

A classifier says there is a bolt in this picture. A detector says here are fifty boxes, and these are the bolts. Dense prediction goes all the way down: it gives a label to every pixel. That is what you need to measure a tumour in a CT slice, map the road surface in an aerial photograph, or count cells that press against each other in a micrograph.

This page covers the networks that do it. Fully convolutional encoder–decoders and U-Net handle semantic segmentation, where each pixel gets a class. Mask R-CNN handles instance segmentation, where each object also gets its own mask. Along the way it covers the upsampling layers every decoder needs, including one output-size trap that catches nearly everyone. The page assumes convolution, pooling and the training loop from CNN from scratch, and region proposals and RoI pooling from the Object Detection page. Further reading is Szeliski (2nd ed.) §5.4.1, §6.4 and §6.4.2.

SIFT, Matching and Stitching

Wed, 11 Nov 2026 21:00:00 +1100

Take two photographs of the same building from slightly different spots. To glue them into one panorama you need three things: points that can be found again in both pictures, a way of deciding which point in one picture is which point in the other, and the single geometric map that carries one picture onto the other even though some of those decisions are wrong. The scale-invariant feature transform (SIFT) of Lowe (2004) supplies the first two; a homography fitted by random sample consensus (RANSAC) supplies the third. 𐃏