Convolutional Neural Networks
Why use CNNs for images instead of dense layers?
A dense layer on a 224×224 image would need hundreds of thousands of weights per neuron (≈150K for an RGB image) and would treat each pixel independently. CNNs exploit two image facts: locality (nearby pixels relate) and translation equivariance (a feature is detected wherever it appears), via small shared filters.
The same small kernel slides over the whole image, so a few weights cover every position.
Each output looks at a small patch, not the entire image — far fewer connections.
A feature is detected wherever it appears, so the net generalizes across location.
- “How many params in a conv layer?” —
(kernel_h × kernel_w × in_channels + 1) × out_channels— independent of image size.
What does the convolution operation do?
A small kernel slides across the image; at each position it does an element-wise multiply-and-sum with the patch beneath it, producing one number in the output feature map. Slide it everywhere and you get a map of where that pattern appears.
- “What do stride and padding control?” — Stride is the step size (bigger → smaller output); padding adds a border so the output keeps the input's size.
- “Output size formula?” —
(W − K + 2P) / S + 1.
What do the filters learn?
They're learned, not hand-designed. Early layers learn simple patterns — edges, colors, textures — and deeper layers compose those into parts and whole objects. A hierarchy of features emerges automatically.
Layer 1: edges & blobs → Layer 2: corners & textures → Layer 3: object parts (eyes, wheels) → deep layers: whole objects. Each filter produces one channel in the feature map.
- “Why multiple filters per layer?” — Each detects a different pattern; their outputs stack into the channel dimension.
- “What is transfer learning here?” — Reuse a pretrained CNN's early filters and fine-tune the last layers for your task.
What is pooling for?
Pooling downsamples the feature map — e.g. max-pool takes the largest value in each 2×2 window. It shrinks spatial size (less compute), adds a bit of translation invariance, and keeps the strongest activations.
- “Max vs average pooling?” — Max keeps the strongest feature (most common); average smooths.
- “Can you skip pooling?” — Yes — strided convolutions can downsample instead; many modern nets use them.
- “Classic architectures?” — LeNet → AlexNet → VGG → ResNet (added residual connections to go very deep).