Convolutional Neural Networks
Convolution & Pooling Operations
A dense/fully-connected layer (Module 2, Chapter 1) connects every input to every neuron. For a small 28×28 grayscale image (784 pixels), a single hidden layer
Jr Codex Deep Learning Notes
Level: Intermediate Prerequisites: Module 3: Training Deep Networks Effectively Time to complete: ~25 minutes
Table of Contents
- Why Dense Layers Fail for Images
- The Convolution Operation
- Filters/Kernels — What They Learn
- Stride and Padding
- Multiple Channels and Multiple Filters
- Pooling
- Convolution and Pooling in PyTorch
- Summary & Next Steps
1. Why Dense Layers Fail for Images
A dense/fully-connected layer (Module 2, Chapter 1) connects every input to every neuron. For a small 28×28 grayscale image (784 pixels), a single hidden layer of just 128 neurons already needs over 100,000 weights — a modest 224×224 color image (roughly 150,000 pixels) would need tens of millions of weights for a single layer alone.
Two Fundamental Problems With Dense Layers on Images
─────────────────────────────────────────
1. PARAMETER EXPLOSION: too many weights to train
efficiently or avoid severe
overfitting (Module 3, Ch.4)
2. NO TRANSLATION a dense layer treats a cat
INVARIANCE: in the top-left corner as
a COMPLETELY different
pattern than the SAME
cat in the bottom-right
— it must re-learn the
same feature separately
for every possible
position
─────────────────────────────────────────
Convolutional layers solve both problems by sharing a small set of weights across the entire image, exploiting the fact that useful visual features (edges, textures) look the same regardless of where they appear.
2. The Convolution Operation
A convolution slides a small filter (or kernel) — a small grid of weights, e.g. 3×3 — across the image, computing a weighted sum at each position.
A 3x3 Filter Sliding Over a Small Image (Conceptual)
─────────────────────────────────────────
Image (5x5): Filter (3x3):
1 2 3 0 1 1 0 -1
4 5 6 1 0 1 0 -1
7 8 9 2 1 1 0 -1
0 1 2 3 4
1 2 3 4 5
At each position, multiply the filter element-wise with the
underlying 3x3 patch of the image, then SUM the results —
producing ONE output value per position. Sliding the filter
across the entire image produces a full output grid, called
a FEATURE MAP.
─────────────────────────────────────────
import numpy as np
def convolve2d(image, kernel):
kernel_h, kernel_w = kernel.shape
output_h = image.shape[0] - kernel_h + 1
output_w = image.shape[1] - kernel_w + 1
output = np.zeros((output_h, output_w))
for i in range(output_h):
for j in range(output_w):
patch = image[i:i+kernel_h, j:j+kernel_w]
output[i, j] = np.sum(patch * kernel) # element-wise multiply, then sum
return output
image = np.array([
[1, 2, 3, 0, 1],
[4, 5, 6, 1, 0],
[7, 8, 9, 2, 1],
[0, 1, 2, 3, 4],
[1, 2, 3, 4, 5],
])
vertical_edge_filter = np.array([
[1, 0, -1],
[1, 0, -1],
[1, 0, -1],
])
feature_map = convolve2d(image, vertical_edge_filter)
print(feature_map)3. Filters/Kernels — What They Learn
Classical computer vision hand-designed filters like the vertical edge detector above. The core innovation of CNNs is that filter values are learned via backpropagation (Module 2, Chapter 4), exactly like weights in a dense layer — nobody tells the network what pattern to detect; it discovers useful filters purely by minimizing the loss.
This Directly Realizes Module 1's "Learned Feature Hierarchy"
─────────────────────────────────────────
Early convolutional layers: learn simple filters —
edges, color blobs,
simple textures
Middle convolutional layers: learn filters responding
to combinations of
early features — corners,
simple shapes, textures
Late convolutional layers: learn filters
responding to
complex, object-
specific patterns —
an eye, a wheel,
a wing
─────────────────────────────────────────
4. Stride and Padding
Two Key Hyperparameters
─────────────────────────────────────────
Stride: how many pixels the filter moves between each
position. Stride 1 (default) slides one pixel
at a time; stride 2 skips every other position,
producing a SMALLER output — used to downsample
while convolving
Padding: adds a border of zeros around the image before
convolving. Without padding, the output
shrinks with EVERY convolution (Section 2's
example: 5x5 input, 3x3 filter → 3x3 output)
— padding ("same" padding) keeps the output
the SAME size as the input, which matters
for building very deep networks (Module 3)
─────────────────────────────────────────
def output_size(input_size, kernel_size, stride=1, padding=0):
return (input_size + 2 * padding - kernel_size) // stride + 1
print(output_size(input_size=5, kernel_size=3, stride=1, padding=0)) # 3 — shrinks
print(output_size(input_size=5, kernel_size=3, stride=1, padding=1)) # 5 — SAME size
print(output_size(input_size=5, kernel_size=3, stride=2, padding=0)) # 2 — downsampled5. Multiple Channels and Multiple Filters
Real images have multiple channels (e.g. Red, Green, Blue), and real convolutional layers use many filters simultaneously, each learning to detect a different pattern.
A Realistic Convolutional Layer
─────────────────────────────────────────
Input: 224 x 224 x 3 (height x width x RGB channels)
Filters: 64 filters, each 3x3x3 (must match the
INPUT channel depth)
Output: 224 x 224 x 64 (one feature map PER
filter — 64 total, assuming "same"
padding, Section 4)
Each of the 64 filters can learn a DIFFERENT pattern —
one might become an edge detector, another a color-blob
detector, another a texture detector — all discovered
automatically during training.
─────────────────────────────────────────
6. Pooling
Pooling reduces a feature map's spatial size, keeping the most important information while discarding precise positional detail — this both reduces computation for later layers and adds a degree of translation invariance.
def max_pool2d(feature_map, pool_size=2, stride=2):
output_h = (feature_map.shape[0] - pool_size) // stride + 1
output_w = (feature_map.shape[1] - pool_size) // stride + 1
output = np.zeros((output_h, output_w))
for i in range(output_h):
for j in range(output_w):
patch = feature_map[i*stride:i*stride+pool_size, j*stride:j*stride+pool_size]
output[i, j] = np.max(patch) # keep only the STRONGEST activation in each region
return output
feature_map = np.array([
[1, 3, 2, 4],
[5, 6, 1, 2],
[2, 1, 8, 3],
[4, 2, 1, 5],
])
pooled = max_pool2d(feature_map, pool_size=2, stride=2)
print(pooled) # [[6, 4], [4, 8]] — the max of each 2x2 regionWhy Pooling Adds Translation Invariance
─────────────────────────────────────────
If a distinctive feature shifts by ONE pixel, MAX POOLING
is likely to still capture the SAME maximum value from the
same general REGION — small positional shifts don't change
the pooled output much, which is exactly the "invariance"
property dense layers lacked (Section 1).
─────────────────────────────────────────
7. Convolution and Pooling in PyTorch
import torch
import torch.nn as nn
conv_layer = nn.Conv2d(
in_channels=3, # RGB input (Section 5)
out_channels=64, # 64 learned filters
kernel_size=3, # 3x3 filters (Section 2)
stride=1, # Section 4
padding=1, # "same" padding (Section 4)
)
pool_layer = nn.MaxPool2d(kernel_size=2, stride=2) # Section 6
sample_image = torch.randn(1, 3, 224, 224) # (batch, channels, height, width)
after_conv = conv_layer(sample_image)
print(f"After conv: {after_conv.shape}") # [1, 64, 224, 224]
after_pool = pool_layer(after_conv)
print(f"After pool: {after_pool.shape}") # [1, 64, 112, 112] — spatial size HALVED8. Summary & Next Steps
Key Takeaways
- Dense layers scale poorly to images due to parameter explosion and a lack of translation invariance; convolutional layers solve both by sharing a small set of learned weights across the entire image.
- A filter/kernel slides across the input, computing a weighted sum at each position to produce a feature map; unlike classical computer vision, these filter values are learned via backpropagation.
- Stride controls how far a filter moves between positions; padding controls whether spatial size shrinks or stays constant.
- Pooling (typically max pooling) reduces spatial size while adding translation invariance, by keeping only the strongest activation in each local region.
Concept Check
- Why does a dense layer applied directly to an image lack translation invariance, and how does a convolutional layer fix this?
- If a 32x32 input is convolved with a 5x5 filter, stride 1, no padding, what is the output size?
- Why does max pooling add a degree of translation invariance?
Next Chapter
→ Chapter 2: CNN Architectures
Jr Codex — 1-on-1 Personalized Coaching | Back to Module Index | Back to DL Index