whatmodel.sh / CogVLM2 Llama3 Chinese Chat 19B

CogVLM2 Llama3 Chinese Chat 19B

+ contribute
Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

CogVLM2 Llama3 Chinese Chat 19B is an open-weights language model from Zhipu AI (THUDM), released May 2024. Its published size is 19.5B parameters. Licence: cogvlm2.

Catalogue identity notes

Comparison-only, proposed so the CogVLM2 card runs of cogvlm2-llama3-chat-19b count. The pinned cogvlm2-llama3-chat-19B card (@f592f29) lists the two own CogVLM2 releases side by side ('cogvlm2-llama3-chat-19B' and 'cogvlm2-llama3-chinese-chat-19B', both Meta-Llama-3-8B-Instruct based, 19B) and scores both in its Benchmark table ('CogVLM2-LLaMA3' and 'CogVLM2-LLaMA3-Chinese'). Weights: https://huggingface.co/zai-org/cogvlm2-llama3-chinese-chat-19B (sha 2c2226281325649d49b8aa237a932367c7da4f26, created 2024-05-16, 19,503,107,328 params in safetensors metadata, licence cogvlm2).

46.6±6.7comparison only: no current run option verifiedfrom 3 benchmark results across 1 source

$ how it compares

The LLMs ranked either side of CogVLM2 Llama3 Chinese Chat 19B, out of 828 with a score, then more from Zhipu AI (THUDM).

#modelscorewhereRAMreleased
483Aya Expanse 8B Cohere47±2local~6.2GB2024-10
484Phind CodeLlama 34B v2 Phind47±1local~23GB2023-08
485CogVLM2 Llama3 Chinese Chat 19B Zhipu AI (THUDM)47±7comparison only2024-05
486InternLM2.5 7B Chat 1M Shanghai AI Laboratory (InternLM)47±4local~6.4GB2024-06
487EuroLLM 9B Instruct UTTER (EuroLLM)47±3local~6.9GB2024-11
more from Zhipu AI (THUDM)
349GLM-4 9B Chat 1M Zhipu AI (THUDM)53±5local~8.1GB2024-06
569CogVLM2 Llama3 Chat 19B Zhipu AI (THUDM)43±3local~43GB2024-05
706ChatGLM3 6B 128K Zhipu AI (THUDM)34±4local~5.8GB2024-01

see all LLMs · find the ones that fit your machine

$ what it's good at

vision & documents▁▁▃▃▃▃▅▄▇▇█▃ #147 of 182 · 80% of its score · 3 tests

Beats DeepSeek V4 Flash Vision (Exp), Gemma 3 12B, Llama 3.2 90B Vision Instruct here, though they rank higher overall.

testresultvs the others in its rundifficultycountswhy
ChartQA lab74.7% ✎▁··▁▁▁▁▁▃▅▅█ beats 8/44easy · avg 81%▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 35%blunt · small run
OCRBench lab78.0% ✎▁·▃▁▆▅▂█▅█▁▃ beats 5/37easy · avg 82%▇▇▇▇▇▇▇▇▇▇▇ 28%blunt · small run
TextVQA lab85.0% ✎▁▁▁▄▅▁▃·▃▃▃█ beats 22/24easy · avg 78%▇▇▇▇▇▇▇ 16%blunt · small run

$ where to run it

No verified API or download recorded yet · add one

$ how the score was worked out · Zhipu AI (THUDM) CogVLM2 mode… 80%

Each result implies a score on its own, after adjusting for how hard its test is and who ran it. The score is their weighted average, with a small pull toward the middle for thinly tested models. Each test's weight is its counts column above; here it's added up by who ran the tests. How it works

sourceresultsshare of the scorelicence
Zhipu AI (THUDM) CogVLM2 model card, Benchmark table (own models) · lab-reported3▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 80%Factual vendor-reported score (not licensed content)
prior · pull toward the middle▇▇▇▇▇▇ 20%
weighted average= 46.6

$ improve this page

Update model informationAdd a place to run itAdd a test resultPropose a benchmarkCorrect a benchmark scoreEdit data on GitHub

Use a short GitHub form or edit the data through a pull request. Every number needs a source; here's which ones we can use.

data updated · sources and licences · json