Global Average Pooling (avgpool) aggregates spatial information by averaging features across the spatial grid:

1. The Operation

The output of the convolutional layer layer4 is a 3D tensor of shape:

$$\text{Shape: } (BatchSize, 512, Height, Width)$$

For a CIFAR-100 image, the spatial grid is $4 \times 4$ (i.e. $Height=4, Width=4$).

AdaptiveAvgPool2d(output_size=(1, 1)) takes the average of all $4 \times 4 = 16$ spatial values for each of the 512 channels independently:

$$\text{avg\_value}_c = \frac{1}{16} \sum_{h=1}^4 \sum_{w=1}^4 \text{activation}_{c, h, w}$$

This collapses the spatial dimensions, leaving a tensor of shape $(BatchSize, 512, 1, 1)$, which is then flattened to $(BatchSize, 512)$.

---

2. Why it is used

• Spatial Invariance: Averaging ensures that the representation is robust to where a feature appears in the image (translation invariance).
• Dimensionality Reduction: It compresses spatial maps into a single vector of 512 numbers that can be fed into the linear classification head (model.fc).
• Preserving Non-negativity: Because it is a simple average of values that are already non-negative (due to the final ReLU of layer4), the averaged result is also strictly non-negative.