Here is the mathematical explanation of how the areas under the curves behave:

1. Relative Weight ($P_{\text{weight}}$) $\rightarrow$ Yes, is mathematically guaranteed to sum to $1.0$

For every single image $j$, we normalize the coefficients so they sum to $1$:

When we average these proportions across all $N$ images to get the relative weight score $P_{\text{weight}}(f_i)$, the sum across all features is:

Because the x-axis grid spacing is $1$ (representing discrete feature indices), the sum of the values equals the area under the curve. Therefore, the area under the $P_{\text{weight}}$ curve is exactly $1.0$.

2. Activation Probability ($P_{\text{active}}$) $\rightarrow$ No, and it shouldn’t be

$P_{\text{active}}(f_i)$ represents the probability that feature $i$ is active in any given image.

In any single image, multiple features are active simultaneously (concepts co-occur, e.g. an image of a car triggers tire, window, metal surface, and street features at the same time).

• If we sum $P_{\text{active}}(f_i)$ across all $K$ features: $$\sum_{i=1}^K P_{\text{active}}(f_i) = \text{Average number of active features per image}$$ For instance, if on average 150 features are active in each image, the sum (and the area under the curve) will be $150$, not $1$.
• Why we do not normalize it to $1.0$: If we forced the area under $P_{\text{active}}$ to sum to $1$, the values would no longer represent absolute probability percentages (between $0.0$ and $1.0$). If normalized to $1$, a feature present in 90% of images would be scaled down to a tiny value like $0.0006$, making it impossible to read its actual frequency of occurrence from the y-axis.

Keeping $P_{\text{active}}$ unnormalized preserves its meaning as a true probability of occurrence (0% to 100%) on the left axis.

Here is the exact code implementation from cifar100_feature_frequency.py showing how both metrics are computed in Python/NumPy:

1. Code Implementation

# Z is the projected activation weight matrix of shape (N, K)
# where N = number of images (e.g. 5000) and K = number of features (2500)

# Method 1: Threshold-based Activation Probability
prob_active = (Z > args.threshold).mean(axis=0)

# Method 2: Normalized Relative Weight Probability
row_sums = Z.sum(axis=1, keepdims=True) + 1e-9
Z_norm = Z / row_sums
prob_weight = Z_norm.mean(axis=0)

2. Line-by-Line Breakdown

A. Computing Activation Probability (prob_active)

1. Z > args.threshold: Generates a boolean mask of shape (N, K). For example:
   • True if feature $i$ is present in image $j$ (above $\tau$, e.g., $10^{-4}$).
   • False if it is inactive or negligible.
2. .mean(axis=0): Averages along the rows (the $N$ images). Since Python treats True as 1 and False as 0, taking the mean computes: $$\frac{\text{Count of True values}}{\text{Total images } N}$$ This yields a vector of length $K$ containing values strictly between $0.0$ and $1.0$.

B. Computing Relative Weight Probability (prob_weight)

1. row_sums = Z.sum(axis=1, keepdims=True) + 1e-9: Computes the sum of all feature weights for each image. This output has shape (N, 1). We add a tiny epsilon (1e-9) to prevent any potential division-by-zero errors.
2. Z_norm = Z / row_sums: Normalizes each image’s feature vector (row) to sum to exactly $1.0$. For example, if an image has three active features with weights [2.0, 3.0, 5.0], they become normalized to [0.2, 0.3, 0.5].
3. Z_norm.mean(axis=0): Averages these normalized values across all $N$ images. The resulting vector of length $K$ represents the average proportion of the model’s representation dedicated to each feature.