MMLU-Pro (TIGER-Lab Leaderboard)
42 models with a score. Leader: Gemini 3.1 Pro (Vorschau) with 91.16 %.
| Benchmark profile | |
|---|---|
| Task | allgemein - Wissen ueber 14 Fachgebiete |
| Metric | Anteil richtiger Antworten, 10 Antwortoptionen, Gesamtwert ueber alle Fachgebiete |
| Direction | ↑ higher is better |
| Limitation | 🔴 GEMISCHTE Quelle: die Rangliste der Benchmark-Autoren fuehrt eigene Messungen UND Herstellerangaben (Spalte Data Source). Der Katalog uebernimmt beide, kennzeichnet aber jeden Wert (mmlu_pro_herkunft) - Herstellerwerte sind ungeprueft. Artificial Analysis hat MMLU-Pro zurueckgezogen; eine unabhaengige Quelle mit aktuellen Modellen fehlt. Das klassische MMLU wird nicht gefuehrt: gesaettigt, und fuer aktuelle Modelle nur noch als Herstellerangabe verfuegbar. |
| As of | 11 Mar 2026 |
| Source | huggingface.co |
| Licence | Apache-2.0 — TIGER-Lab MMLU-Pro |
Ranking (42)
No entry matches the filter.
| Model | Rank | Score | Developer | EU processing: InputEU in | EU processing: OutputEU out | all providers: Inputall in | all providers: Outputall out | Open weights | GPU memory (4-bit, derived)GPU memory |
|---|---|---|---|---|---|---|---|---|---|
| Gemini 3.1 Pro (Vorschau) | 1 | 91.16 % | – | – | €0.90 | €5.37 | – no | – | |
| Qwen3.5-397B-A17B | 2 | 87.8 % | Alibaba Cloud | €0.60 | €3.58 | €0.40 | €2.69 | ✓ yes | from 226 GB |
| GPT-5.4 | 3 | 87.5 % | OpenAI | €2.60 | €13.87 | €1.12 | €6.72 | – no | – |
| Gemini 3.1 Flash-Lite | 4 | 86.2 % | €0.24 | €1.46 | €0.11 | €0.67 | – no | – | |
| Qwen3.5-27B | 5 | 86.1 % | Alibaba Cloud | – | – | €0.23 | €2.33 | ✓ yes | from 20 GB |
| Gemini 2.5 Pro | 6 | 86 % | €1.34 | €8.94 | €0.56 | €4.48 | – no | – | |
| GLM-5 | 6 | 86 % | Z.ai | €1.15 | €3.68 | €0.45 | €1.43 | – | – |
| Qwen3.5-9B | 8 | 82.5 % | Alibaba Cloud | €0.09 | €0.13 | €0.09 | €0.13 | ✓ yes | from 10 GB |
| GPT-4.1 | 9 | 81.8 % | OpenAI | €1.97 | €7.87 | €0.90 | €3.58 | – no | – |
| gpt-oss-120b | 10 | 80.8 % | OpenAI | €0.026 | €0.12 | €0.026 | €0.12 | ✓ yes | from 69 GB |
| MiniMax M2.5 | 11 | 80.1 % | MiniMax | €0.31 | €1.25 | €0.13 | €0.54 | ✓ yes | from 130 GB |
| Claude 3.5 Sonnet v2 | 12 | 77.64 % | Anthropic | – | – | €1.34 | €6.72 | – | – |
| Claude 3.5 Sonnet | 13 | 76.12 % | Anthropic | – | – | €1.34 | €6.72 | – | – |
| GPT-4o | 14 | 74.7 % | OpenAI | €2.39 | €9.55 | €1.12 | €4.48 | – no | – |
| GPT-4o-2024-08-06 | 15 | 74.68 % | OpenAI | €1.23 | €4.93 | €1.12 | €4.48 | – no | – |
| gpt-oss-20b | 16 | 73.1 % | OpenAI | €0.024 | €0.11 | €0.024 | €0.11 | ✓ yes | from 16 GB |
| GPT-4o-2024-0513 | 17 | 72.55 % | OpenAI | €4.93 | €14.78 | €2.24 | €6.72 | – no | – |
| Phi-4 | 18 | 70.4 % | Microsoft | – | – | €0.063 | €0.13 | ✓ yes | from 13 GB |
| Claude 3 Opus | 19 | 68.45 % | Anthropic | – | – | €6.72 | €33.59 | – | – |
| Mistral Large 2.1 | 20 | 67.9 % | Mistral AI | – | – | – | – | ✓ yes | from 72 GB |
| Gemma 3 27B | 21 | 67.5 % | €0.089 | €0.27 | €0.072 | €0.14 | ✓ yes | from 20 GB | |
| Mistral Small 3 24B | 22 | 66.3 % | Mistral AI | – | – | €0.045 | €0.072 | ✓ yes | from 17 GB |
| Llama 3.3 70B | 23 | 65.9 % | Meta | €0.45 | €0.65 | €0.09 | €0.29 | ✓ yes | from 43 GB |
| GPT-4-Turbo | 24 | 63.71 % | OpenAI | – | – | €10.84 | €32.51 | – no | – |
| GPT-4o mini | 25 | 63.1 % | OpenAI | €0.14 | €0.57 | €0.067 | €0.27 | – no | – |
| GPT-4o-mini-0718 | 26 | 63.09 % | OpenAI | €0.15 | €0.59 | €0.067 | €0.27 | – no | – |
| Llama 3.1 70B | 27 | 62.84 % | Meta | €0.31 | €1.25 | €0.31 | €1.25 | ✓ yes | from 43 GB |
| Claude 3.5 Haiku | 28 | 62.12 % | Anthropic | – | – | €0.36 | €1.79 | – no | – |
| Gemma 3 12B | 29 | 60.6 % | – | – | €0.045 | €0.13 | ✓ yes | from 11 GB | |
| Gemini 1.5 Flash | 30 | 59.12 % | – | – | – | – | – no | – | |
| Claude 3 Sonnet | 31 | 56.8 % | Anthropic | – | – | €1.34 | €6.72 | – no | – |
| Gemma 2 9B | 32 | 52.1 % | – | – | – | – | ✓ yes | from 10 GB | |
| Jamba 1.5 Large | 33 | 49.46 % | AI21 Labs | – | – | €1.79 | €7.17 | – | – |
| Aya Expanse 32B | 34 | 45.41 % | – | – | – | €0.45 | €1.34 | – | – |
| Mistral Nemo Instruct | 35 | 44.81 % | Mistral AI | €0.016 | €0.025 | €0.016 | €0.025 | ✓ yes | from 11 GB |
| Llama 3.1 8B Instruct | 36 | 44.25 % | Meta | €0.016 | €0.041 | €0.018 | €0.036 | ✓ yes | from 9 GB |
| Mixtral 8x7B Instruct | 37 | 43.27 % | Mistral AI | €0.44 | €0.68 | €0.40 | €0.63 | ✓ yes | from 30 GB |
| Claude 3 Haiku | 38 | 42.29 % | Anthropic | – | – | €0.11 | €0.56 | – no | – |
| WizardLM 2 8x22B | 39 | 39.24 % | – | – | – | €0.56 | €0.56 | ✓ yes | – |
| Command R 03-2024 | 40 | 37.9 % | Cohere | – | – | €0.45 | €1.34 | ✓ yes | from 24 GB |
| Aya Expanse 8B | 41 | 33.74 % | – | – | – | €0.45 | €1.34 | – | – |
| mistral-7b-instruct-v0.2 | 42 | 30.84 % | Mistral AI | €0.14 | €0.20 | €0.14 | €0.20 | ✓ yes | from 8 GB |
Equal scores share a rank. When filtering, the rank stays the position in the whole list. GPU memory “from …” is a lower bound (parameters × 0.55 GB + 4 GB). Prices: € per 1M tokens. Source: providers' pricing pages, read and checked by searchit. As of 7 Nov 2026.
Price and performance
Scores: MMLU-Pro (TIGER-Lab Leaderboard), Apache-2.0 — TIGER-Lab MMLU-Pro. Taken unchanged from the source.