4.1 Architecture
CAM++ [8] is a speaker verification model developed by Alibaba’s DAMO Academy, available through the 3D-
Speaker project on HuggingFace. The architecture:
80-band FBANK
↓FCM (Front-End Convolutional Module)
– Conv2D (channels: 1 → 64)
– 2× ResBlock (64 channels)
↓
TDNN (kernel=5, channels: 64 → 128)
↓
3× DenseTDNN Blocks
– Block 1: 12 layers
– Block 2: 24 layers
– Block 3: 16 layers
Each block uses context-aware masking
↓
Attentive Statistical Pooling
– Mean + Standard deviation pooling
– Global + segment-level attention
↓
Dense Layer → 192-dim embedding
↓
L2 Normalization → ‖v‖ = 1
Total parameters: approximately 12 million.
4.2 Speaker Verification Pretraining
CAM++ is trained on the VoxCeleb dataset (over 200k speakers from YouTube videos) with aggressive data
augmentation, including: – Additive noise (babble, music, noise) – Room impulse response convolution – Spectral
augmentation (frequency masking) – SpecAugment time-frequency masking – Random equalization and filtering
4.3 The Invariance Mechanism
The spectral augmentation is critical: it trains the model to be invariant to channel effects, equalization, and spectral
coloration. This is achieved through several mechanisms:
1.Frequency masking — Random bands of the spectrogram are masked during training, forcing the model to rely
on speaker-specific characteristics that persist across spectral perturbations.
2.Style shortcut avoidance — The DenseTDNN blocks with context-aware masking prevent the model from
fixating on narrow frequency bands that might encode channel information rather than speaker identity.
3.Attentive pooling — The pooling layer attends to frame-level features that are most speaker-discriminative while
down-weighting channel-dependent features.
4.Metric learning objective — The additive margin softmax loss encourages inter-class (between-speaker)
separation while intra-class (within-speaker) variance — including channel variance — is minimized.
This is the correct behavior for speaker verification — the model should recognize the same speaker regardless of
recording conditions. However, for our perceptual style extraction task, it presents a fundamental challenge: the
embedding space is deliberately compressed to discard the very spectral information we aim to capture.
4.4 Role in Seed-VC
In Seed-VC, CAM++ is used as the style encoder: – style_as_token=True: the 192-dim embedding is projected and
concatenated with content tokens – Class-dropout during training enables CFG control over style strength – The style
embedding is injected into the DiT conditioning via cross-attention
4.5 Why CAM++?
CAM++ was selected as the speaker encoder for three reasons:
- Native Seed-VC integration. CAM++ is the default style encoder in Seed-VC, enabling direct injection of ΔV
into the style conditioning pipeline without any architectural modifications. Using any other encoder would
require re-engineering the core conditioning mechanism.
2.Proven performance. Trained on over 200,000 speakers (VoxCeleb) with aggressive data augmentation, CAM++
achieves state-of-the-art speaker verification accuracy. Its 192-dimensional L2-normalized embeddings are well-
understood, widely benchmarked, and mathematically clean for delta vector computation.3.
Full reproducibility. The model is openly available on HuggingFace through Alibaba’s 3D-Speaker project. All
results in this study can be independently verified and extended.
The key limitation spectral invariance to bone-conduction filtering was not known at the time of selection. It
was discovered through this project and is presented as Finding 1.
