The loss function minimized by the optimizer is:

$$\mathcal{L} = -\left(S_{\text{mean}} - \alpha \cdot S_{\text{spread}}\right) + w_{\text{norm}} \cdot N_{\text{penalty}}$$

Let’s break this down:

1. The Core Objective (to maximize)

We want to maximize $S_{\text{mean}} - \alpha \cdot S_{\text{spread}}$.

• S_mean: The average cosine similarity of the 3 sentences to the target profile: $$S_{\text{mean}} = \frac{S_1 + S_2 + S_3}{3}$$
• S_spread: The gap between the average similarity and the worst similarity: $$S_{\text{spread}} = S_{\text{mean}} - \min(S_1, S_2, S_3)$$ This penalizes a candidate if it performs well on one sentence but terribly on another, driving the style vectors to generalize robustly.

Substituting the default $\alpha = 0.4$, the objective simplifies to:

$$\text{Objective} = 0.6 \cdot S_{\text{mean}} + 0.4 \cdot S_{\text{min}}$$

To maximize this objective, the optimizer minimizes its negative:

$$\mathcal{L}_{\text{similarity}} = -\left(0.6 \cdot S_{\text{mean}} + 0.4 \cdot S_{\text{min}}\right)$$

---

2. The Manifold Regularization (to minimize)

• N_penalty: The squared z-score deviation of the style row L2 norms from the stock voices: $$N_{\text{penalty}} = \frac{1}{|U|} \sum_{r \in U} \left(\frac{\|V_r\|_2 - M_r}{SD_r}\right)^2$$ where $U = \{42, 47, 48\}$ are the active phoneme rows. This regularizes the optimization so that the style vectors stay within the natural volume boundary of human voices.

---

Final Combined Loss:

With default weights:

$$\mathcal{L} = -\left(0.6 \cdot S_{\text{mean}} + 0.4 \cdot S_{\text{min}}\right) + 0.1 \cdot N_{\text{penalty}}$$