whatmodel.sh / GPT-4o Audio Preview (2024-10-01)
GPT-4o Audio Preview (2024-10-01)
↗ share
+ contribute
GPT-4o Audio Preview (2024-10-01) is a closed-weights language model from OpenAI, released October 2024. Licence: proprietary.
Catalogue identity notes
Comparison-only. OpenAI audio-input API snapshot 'gpt-4o-audio-preview-2024-10-01' (exact identifier and release date in AHELM Table 3, arXiv 2508.21376v2). Distinct from the text GPT-4o snapshots. No run route or price recorded.
$ how it compares
The LLMs ranked either side of GPT-4o Audio Preview (2024-10-01), out of 828 with a score, then more from OpenAI.
| # | model | score | where | ||
|---|---|---|---|---|---|
| 528 |  QwQ-32B-Preview | 45±2 | local | ||
| 529 |  DeepSeek-Coder-V2 Lite Instruct (16B-A2.4B) | 45±1 | local | ||
| 530 | GPT-4o Audio Preview (2024-10-01) | 45±3 | comparison only | ||
| 531 |  Claude 2.1 | 45±1 | comparison only | ||
| 532 |  Zephyr ORPO 141B-A35B v0.1 | 45±2 | local | ||
| more from OpenAI | |||||
| 1 | GPT-6 Astra | 87±1 | api $20.00/M | ||
| 3 | GPT-6.1 Sol | 85±1 | api $4.00/M | ||
| 7 | GPT-6 Sol | 83±1 | api $4.00/M | ||
$ what it's good at
other tests10 not in a task · 97% of its score
| test | result | vs the others in its run | counts |
|---|---|---|---|
| AHELM Speech Robust Bench (LibriSpeech-clean with noise) transcription WER | 82.2% ✎ | ▃·█··▆▄·▂▂·▄ beats 0/13 | 51% |
| AHELM CoRe-Bench conversational audio reasoning (pseudo-exact match) | 34.2% ✎ | ▂·▄··▂▂·▄▂█▄ beats 1/13 | 18% |
| AHELM LibriSpeech transcription by audio LLMs (WER) | 11.3% ✎ | ▃···▃·█▃▅▃▅█ beats 3/13 | 10% |
| AHELM MUStARD audio sarcasm detection (exact match) | 59.3% ✎ | ▂·······▆█▆▆ beats 8/13 | 5.2% |
| AHELM AIR-Bench Foundation music subsets (exact match) | 52.7% ✎ | ▃█▃▅▃·▃·▃▅▃▃ beats 3/13 | 4.6% |
| AHELM VoxCeleb2 same-speaker detection (exact match) | 57.0% ✎ | ▂·▂·▂█▂▄▄··▄ beats 8/13 | 3.9% |
| AHELM MELD audio emotion classification (pseudo-exact match) | 45.6% ✎ | ▃··▃▃▃·▃▅▅█▅ beats 7/13 | 2.6% |
| AHELM FLEURS 7-language transcription by audio LLMs (WER) | 77.1% ✎ | ▄▂···▃▆▂█▂·▂ beats 3/13 | 1.6% |
2 more tests
| test | result | vs the others in its run | counts |
|---|---|---|---|
| AHELM Multilingual LibriSpeech 7-language transcription by audio LLMs (WER) | 16.2% ✎ | ▂▂▂··▂▃·▂▂█▆ beats 4/13 | 0.2% |
| AHELM VocalSound human-sound recognition (prefix exact match) | 83.3% ✎ | ██▄▄·█·█·█▄▄ beats 10/13 | 0.1% |
$ where to run it
No verified API or download recorded yet · add one
$ how the score was worked out · AHELM: A Holistic Evaluation… 97%
Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works
| source | results | share of the score |
|---|---|---|
 AHELM: A Holistic Evaluation of Audio-Language Models (Stanford CRFM, arXiv:2508.21376v2) | 10 | 97% |
| prior · pull toward the middle | 3% | |
| weighted average | = 45.0 |
$ improve this page
Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.
data updated · sources and licences · json