whatmodel.sh / tts / Amphion VALL-E v2 recipe (MLS)
Amphion VALL-E v2 recipe (MLS)
↗ share
+ contribute
Amphion VALL-E v2 recipe (MLS) is an open-weights text-to-speech model from Amphion (CUHK-Shenzhen), released June 2024. Licence: mit.
Catalogue identity notes
amphion/valle (valle_ar_mls_196000.bin + valle_nar_mls_164000.bin with SpeechTokenizer): Amphion's VALL-E v2-recipe reproduction. Not Microsoft's VALL-E 2 (unreleased). Card has no licence tag; Amphion repository is MIT.
Score within the Vapi Humanness · Bradley–Terry rating, 2026-09-25 snapshot comparison set only.
$ how it compares
The text-to-speech models ranked either side of Amphion VALL-E v2 recipe (MLS), out of 148 with a score, then more from Amphion (CUHK-Shenzhen).
| # | model | score | where | ||
|---|---|---|---|---|---|
| 119 |  Gemini 3.8 Flash-Lite TTS | 37±3 | api price unknown | ||
| 120 |  Eleven v4 | 37±4 | api $80.00/M chars | ||
| 121 | Amphion VALL-E v2 recipe (MLS) | 36±3 | local | ||
| 122 |  NeuTTS Air | 36±10 | local | ||
| 123 | Eleven v4 Turbo | 36±4 | api $40.00/M chars | ||
| more from Amphion (CUHK-Shenzhen) | |||||
| 53 | MaskGCT | 59±1 | local | ||
| 95 | Amphion VALL-E (LibriLight 6k) | 47±14 | local | ||
| 97 | Vevo (Amphion) | 46±3 | local | ||
see all text-to-speech models · find the ones that fit your machine
$ what it's good at
other tests6 not in a task · 97% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| TTSDS2 listening test · speaker-similarity MOS (4 English domains) | 3.43 / 5 ✎ | ▅·█▅▅·█▃▃▅·▅ beats 10/17 | 64% |
| VERSA TTS demo · LibriSpeech test-clean UTMOS | 3.65 / 5 ✎ | ▄·····▄·▄█·▄ beats 2/5 | 21% |
| TTSDS2 listening test · naturalness MOS (4 English domains) | 2.90 / 5 ✎ | ▅··▃▅··▅█▅██ beats 6/17 | 5.6% |
| TTSDS2 multilingual benchmark · English overall (YouTube, 2025 round) | 82.1% ✎ | ▂▂█·▂·█▆▂·▄▂ beats 2/17 | 4.1% |
| TTSDS v1 · default four-factor score | 70.3% ✎ | ▃▂····▃·▃▅▂█ beats 0/14 | 0.8% |
| VERSA TTS demo · LibriSpeech test-clean Whisper WER | 27.8% ✎ | ▄··········█ beats 0/5 | 0.6% |
$ where to run it
Other ways to run it
- Run it as published · needs ~2.86 GB · from github.com/open-mmlab
git clone https://github.com/open-mmlab/Amphion.git && cd Amphion && git checkout 6b1411c7a415af40a775dc940c3b29fc18d37970 && conda create --name amphion python=3.9.15 && conda activate amphion && sh env.sh && pip install -U transformers==4.41.2 && huggingface-cli download amphion/valle valle_ar_mls_196000.bin valle_nar_mls_164000.bin --local-dir ckpts && mkdir -p ckpts/speechtokenizer_hubert_avg && huggingface-cli download amphion/valle SpeechTokenizer.pt config.json --local-dir ckpts/speechtokenizer_hubert_avg && python -c "import librosa, torch, torchaudio; from models.tts.valle_v2.valle_inference import ValleInference; from models.tts.valle_v2.g2p_processor import G2pProcessor; model = ValleInference(ar_path='ckpts/valle_ar_mls_196000.bin', nar_path='ckpts/valle_nar_mls_164000.bin', speechtokenizer_path='ckpts/speechtokenizer_hubert_avg', device='cpu'); wav, _ = librosa.load('./egs/tts/VALLE_V2/example.wav', sr=16000); wav = torch.tensor(wav, dtype=torch.float32); g2p = G2pProcessor(); prompt_transcript = torch.tensor(g2p('and keeping eternity before the eyes', 'en')[1]).long(); target_transcript = torch.tensor(g2p('Hello, world.', 'en')[1]).long(); batch = {'speech': wav.unsqueeze(0), 'phone_ids': torch.cat([prompt_transcript, target_transcript], dim=-1).unsqueeze(0)}; output_wav = model(batch, [dict(top_p=0.9, top_k=5, temperature=0.95, repeat_penalty=1.0, max_length=2000, num_beams=1)]); torchaudio.save('out.wav', output_wav.squeeze(0), 16000)"
Official weights and documentation: huggingface.co/amphion ↗
$ how the score was worked out · TTSDS2 listening test results… 70%, VERSA toolkit paper: TTS demo… 22%, TTSDS2 multilingual benchmark… 4% and 1 more
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
| source | results | share of the score |
|---|---|---|
 TTSDS2 listening test results (ttsds/listening_test) | 2 | 70% |
 VERSA toolkit paper: TTS demonstration (arXiv:2412.17667v2) | 2 | 22% |
| TTSDS2 multilingual benchmark results (ttsds/leaderboard_data) | 1 | 4% |
| TTSDS v1 published results | 1 | 1% |
| prior · pull toward the middle | 3% | |
| weighted average | = 36.3 |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json