whatmodel.sh / GPT-4o mini Audio Preview (2024-12-17)
GPT-4o mini Audio Preview (2024-12-17)
↗ share
+ contribute
GPT-4o mini Audio Preview (2024-12-17) is a closed-weights language model from OpenAI, released December 2024. Licence: proprietary.
Catalogue identity notes
Comparison-only. OpenAI audio-input API snapshot 'gpt-4o-mini-audio-preview-2024-12-17' (exact identifier and release date in AHELM Table 3, arXiv 2508.21376v2). Distinct from the text GPT-4o snapshots. No run route or price recorded.
$ how it compares
The LLMs ranked either side of GPT-4o mini Audio Preview (2024-12-17), out of 828 with a score, then more from OpenAI.
| # | model | score | where | ||
|---|---|---|---|---|---|
| 457 |  Phi-3.5-MoE-instruct | 48±1 | local | ||
| 458 |  Claude 3 Haiku | 48±0 | comparison only | ||
| 459 | GPT-4o mini Audio Preview (2024-12-17) | 48±2 | comparison only | ||
| 460 |  Gemma 3n E4B | 48±1 | local | ||
| 461 |  Hermes 2 Theta Llama-3 70B | 48±2 | local | ||
| more from OpenAI | |||||
| 1 | GPT-6 Astra | 87±1 | api $20.00/M | ||
| 3 | GPT-6.1 Sol | 85±1 | api $4.00/M | ||
| 7 | GPT-6 Sol | 83±1 | api $4.00/M | ||
$ what it's good at
other tests10 not in a task · 97% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| AHELM Speech Robust Bench (LibriSpeech-clean with noise) transcription WER | 47.1% ✎ | ▂·█··▆▄·▂▂·▄ beats 2/13 | 50% |
| AHELM CoRe-Bench conversational audio reasoning (pseudo-exact match) | 51.4% ✎ | ▂·▄··▃▂·▄▂█▄ beats 3/13 | 18% |
| AHELM LibriSpeech transcription by audio LLMs (WER) | 16.3% ✎ | ▃···▃·█▃▅▃▅█ beats 1/13 | 10% |
| AHELM MUStARD audio sarcasm detection (exact match) | 62.3% ✎ | ▂·······▆█▆▆ beats 11/13 | 5.7% |
| AHELM VoxCeleb2 same-speaker detection (exact match) | 50.9% ✎ | ▂·▃·▂█▂▄▄··▄ beats 1/13 | 5.0% |
| AHELM AIR-Bench Foundation music subsets (exact match) | 54.1% ✎ | ▃█▃▅▃·▃·▃▅▃▃ beats 4/13 | 4.4% |
| AHELM MELD audio emotion classification (pseudo-exact match) | 33.4% ✎ | ▃··▃▃▃·▃▅▅█▅ beats 1/13 | 2.6% |
| AHELM FLEURS 7-language transcription by audio LLMs (WER) | 54.5% ✎ | ▄▂···▂▆▃█▂·▂ beats 7/13 | 1.6% |
2 more tests
| test | result | vs the others in its run | counts |
|---|---|---|---|
| AHELM Multilingual LibriSpeech 7-language transcription by audio LLMs (WER) | 12.3% ✎ | ▂▂▂··▂▂·▃▂█▆ beats 5/13 | 0.2% |
| AHELM VocalSound human-sound recognition (prefix exact match) | 79.4% ✎ | ██▄▄·█·█·█▄▄ beats 8/13 | 0.1% |
$ where to run it
No verified API or download recorded yet · add one
$ how the score was worked out · AHELM: A Holistic Evaluation… 97%
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
| source | results | share of the score |
|---|---|---|
 AHELM: A Holistic Evaluation of Audio-Language Models (Stanford CRFM, arXiv:2508.21376v2) | 10 | 97% |
| prior · pull toward the middle | 3% | |
| weighted average | = 47.8 |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json