Part 2 · Neural Networks
Convolutional Networks and Vision
Vision networks scan an image for small patterns, then combine them into larger and larger ideas.
Key concepts
- A convolution slides a small filter across an image and records where its pattern appears.
- Pooling and strides summarize larger areas.
- Sharing one filter across the image greatly reduces the number of parameters.
- Vision Transformers split images into patches and use attention (Chapter 15).
- Adversarial examples are small, deliberate changes that can fool a model.
A convolutional network builds larger visual features from smaller ones.
Formal view and evidence
- Established. Convolution layers are translation-equivariant, up to boundary and sampling effects: shifting the input shifts the feature map. Pooling adds limited invariance.
- Debated. The relative roles of architecture, data scale, and augmentation in comparing CNNs and Vision Transformers. ImageNet CNNs have shown texture bias, which depends on training data and augmentation.
- Open problem. Robust perception that matches human reliability under natural and adversarial shifts.
Common misconception
Common belief
"Vision AI sees objects like we do."
In practice
A model may rely on textures or background cues that people ignore.
Key takeaways
- Convolutions reuse the same filter across an image.
- Layers build complex visual features from simple ones.
- Vision models can rely on cues that people would not use.