<!-- https://models.searchit-agent.ai/en/leaderboards/hhem/ | searchit Models -->

# Vectara HHEM-2.3

82 models with a score. Leader: GPT-5.4 nano with 3.1 %.

| Benchmark profile |  |
|---|---|
| Task | Zusammenfassung - Treue zur Vorlage |
| Metric | Anteil der Zusammenfassungen mit mindestens einer Aussage, die im Quelldokument nicht steht |
| Direction | ↓ lower is better |
| Limitation | Nur Englisch, und die Antwortrate gehoert zur Zahl: wer ein Drittel der Dokumente verweigert, halluziniert auf dem Rest seltener. Gemessen wird Treue zur Vorlage, nicht Nuetzlichkeit - ein Modell, das abschreibt, gewinnt. |
| As of | 11 May 2026 |
| Source | [github.com](https://github.com/vectara/hallucination-leaderboard) |
| Licence | Apache-2.0 — Vectara Hallucination Leaderboard |

## Ranking (82)

| Model | Rank | Score | Answer rate | Developer | EU processing: Input | EU processing: Output | all providers: Input | all providers: Output | Open weights | GPU memory (4-bit, derived) |
|---|---:|---:|---:|---|---:|---:|---:|---:|---|---:|
| [GPT-5.4 nano](https://models.searchit-agent.ai/en/models/gpt-5-4-nano/) | 1 | **3.1 %** | 100 % | OpenAI | – | – | €0.09 | €0.56 | – no | – |
| [Gemini 2.5 Flash-Lite](https://models.searchit-agent.ai/en/models/gemini-2-5-flash-lite/) | 2 | **3.3 %** | 99.5 % | Google | – | – | €0.045 | €0.18 | – no | – |
| [Phi-4](https://models.searchit-agent.ai/en/models/phi-4/) | 3 | **3.7 %** | 80.7 % | Microsoft | – | – | €0.063 | €0.13 | ✓ yes | [from 13 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-24-gb) |
| [Llama 3.3 70B](https://models.searchit-agent.ai/en/models/llama-3-3-70b/) | 4 | **4.1 %** | 99.5 % | Meta | €0.45 | €0.65 | €0.09 | €0.29 | ✓ yes | [from 43 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-48-gb) |
| [Gemma 3 12B](https://models.searchit-agent.ai/en/models/gemma-3-12b/) | 5 | **4.4 %** | 97.4 % | Google | – | – | €0.045 | €0.13 | ✓ yes | [from 11 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-24-gb) |
| [Mistral Large 2.1](https://models.searchit-agent.ai/en/models/mistral-large-2-1/) | 6 | **4.5 %** | 99.9 % | Mistral AI | – | – | – | – | ✓ yes | [from 72 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-96-gb) |
| [Qwen3 8B](https://models.searchit-agent.ai/en/models/qwen3-8b/) | 7 | **4.8 %** | 99.9 % | Alibaba Cloud | – | – | – | – | ✓ yes | [from 9 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-24-gb) |
| [Mistral Small 3 24B](https://models.searchit-agent.ai/en/models/mistral-small-3-24b/) | 8 | **5.1 %** | 97.9 % | Mistral AI | – | – | €0.045 | €0.072 | ✓ yes | [from 17 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-24-gb) |
| [Nova 2 Lite](https://models.searchit-agent.ai/en/models/nova-2-lite/) | 8 | **5.1 %** | 99.6 % | Amazon | €0.34 | €2.82 | €0.15 | €1.23 | – no | – |
| [Nova Pro](https://models.searchit-agent.ai/en/models/nova-pro/) | 8 | **5.1 %** | 99.3 % | Amazon | €0.82 | €3.29 | €0.36 | €1.43 | – no | – |
| [Gemma 4 26B A4B](https://models.searchit-agent.ai/en/models/gemma-4-26b-a4b/) | 11 | **5.2 %** | 99.8 % | Google | €0.11 | €0.27 | €0.063 | €0.30 | – | – |
| [Granite 4.0 H Small](https://models.searchit-agent.ai/en/models/granite-4-0-h-small/) | 11 | **5.2 %** | 100 % | IBM | – | – | – | – | ✓ yes | [from 22 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-24-gb) |
| [Qwen3-14B](https://models.searchit-agent.ai/en/models/qwen3-14b/) | 13 | **5.4 %** | 99.9 % | Alibaba Cloud | €0.18 | €0.90 | €0.11 | €0.21 | ✓ yes | [from 13 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-24-gb) |
| [DeepSeek V3.1](https://models.searchit-agent.ai/en/models/deepseek-v3-1/) | 14 | **5.5 %** | 94.5 % | DeepSeek | – | – | €0.22 | €0.85 | ✓ yes | [from 374 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-4x) |
| [GPT-5.4 mini](https://models.searchit-agent.ai/en/models/gpt-5-4-mini/) | 14 | **5.5 %** | 100 % | OpenAI | €0.74 | €4.43 | €0.34 | €2.02 | – no | – |
| [Nova Micro](https://models.searchit-agent.ai/en/models/nova-micro/) | 14 | **5.5 %** | 100 % | Amazon | €0.036 | €0.14 | €0.031 | €0.13 | – no | – |
| [GPT-4.1](https://models.searchit-agent.ai/en/models/gpt-4-1/) | 17 | **5.6 %** | 99.9 % | OpenAI | €1.97 | €7.87 | €0.90 | €3.58 | – no | – |
| [GPT-4.1-2025-04-14](https://models.searchit-agent.ai/en/models/gpt-4-1-2025-04-14/) | 17 | **5.6 %** | 99.9 % | OpenAI | €0.99 | €3.94 | €0.90 | €3.58 | – no | – |
| [Qwen3-32B](https://models.searchit-agent.ai/en/models/qwen3-32b/) | 19 | **5.9 %** | 99.9 % | qwen-alibaba | €0.28 | €0.28 | €0.072 | €0.25 | ✓ yes | [from 23 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-24-gb) |
| [DeepSeek-V3](https://models.searchit-agent.ai/en/models/deepseek-v3/) | 20 | **6.1 %** | 97.5 % | DeepSeek | – | – | €0.29 | €0.80 | ✓ yes | [from 381 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-4x) |
| [Nova Lite](https://models.searchit-agent.ai/en/models/nova-lite/) | 20 | **6.1 %** | 99.9 % | Amazon | €0.062 | €0.25 | €0.054 | €0.21 | – no | – |
| [deepseek-v3.2](https://models.searchit-agent.ai/en/models/deepseek-v3-2/) | 22 | **6.3 %** | 92.6 % | DeepSeek | €0.35 | €0.58 | €0.23 | €0.34 | ✓ yes | [from 381 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-4x) |
| [Gemma 3 4B](https://models.searchit-agent.ai/en/models/gemma-3-4b/) | 23 | **6.4 %** | 67.3 % | Google | – | – | €0.018 | €0.036 | ✓ yes | [from 7 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-24-gb) |
| [GPT-6 sol](https://models.searchit-agent.ai/en/models/gpt-6-sol/) | 24 | **6.5 %** | 100 % | OpenAI | €2.15 | €10.75 | €0.90 | €4.48 | – no | – |
| [Gemini 2.5 Pro](https://models.searchit-agent.ai/en/models/gemini-2-5-pro/) | 25 | **7 %** | 99.1 % | Google | €1.34 | €8.94 | €0.56 | €4.48 | – no | – |
| [GPT-5.4](https://models.searchit-agent.ai/en/models/gpt-5-4/) | 25 | **7 %** | 99.9 % | OpenAI | €2.60 | €13.87 | €1.12 | €6.72 | – no | – |
| [Gemma 3 27B](https://models.searchit-agent.ai/en/models/gemma-3-27b/) | 27 | **7.4 %** | 98.8 % | Google | €0.089 | €0.27 | €0.072 | €0.14 | ✓ yes | [from 20 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-24-gb) |
| [Gemma 4 31B](https://models.searchit-agent.ai/en/models/gemma-4-31b/) | 27 | **7.4 %** | 92.6 % | Google | €0.09 | €0.27 | €0.09 | €0.27 | ✓ yes | [from 22 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-24-gb) |
| [Llama 4 Scout 17B-16E](https://models.searchit-agent.ai/en/models/llama-4-scout/) | 29 | **7.7 %** | 99 % | Meta | – | – | €0.09 | €0.27 | ✓ yes | [from 64 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-96-gb) |
| [Gemini 2.5 Flash](https://models.searchit-agent.ai/en/models/gemini-2-5-flash/) | 30 | **7.8 %** | 99 % | Google | €0.27 | €2.24 | €0.13 | €1.12 | – no | – |
| [Gemini 3.1 Flash-Lite](https://models.searchit-agent.ai/en/models/gemini-3-1-flash-lite/) | 31 | **8.2 %** | 99.6 % | Google | €0.24 | €1.46 | €0.11 | €0.67 | – no | – |
| [Llama 4 Maverick](https://models.searchit-agent.ai/en/models/llama-4-maverick-17b/) | 31 | **8.2 %** | 100 % | Meta | – | – | €0.16 | €0.52 | ✓ yes | [from 225 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-4x) |
| [GPT-5.4 Pro](https://models.searchit-agent.ai/en/models/gpt-5-4-pro/) | 33 | **8.3 %** | 100 % | OpenAI | – | – | €13.43 | €80.61 | – no | – |
| [DeepSeek V4 Pro](https://models.searchit-agent.ai/en/models/deepseek-v4-pro/) | 34 | **8.6 %** | 97.2 % | DeepSeek | €0.90 | €1.79 | €0.90 | €1.79 | ✓ yes | [from 884 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-rechenzentrum) |
| [MiniMax M2.5](https://models.searchit-agent.ai/en/models/minimax-m2-5/) | 35 | **9.1 %** | 98.2 % | MiniMax | €0.31 | €1.25 | €0.13 | €0.54 | ✓ yes | [from 130 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-2x) |
| [GLM 4.7 Flash](https://models.searchit-agent.ai/en/models/glm-4-7-flash/) | 36 | **9.3 %** | 91.6 % | Z.ai | €0.072 | €0.43 | €0.031 | €0.18 | ✓ yes | [from 21 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-24-gb) |
| [GPT-5.5](https://models.searchit-agent.ai/en/models/gpt-5-5/) | 36 | **9.3 %** | 100 % | OpenAI | €4.93 | €29.56 | €2.24 | €13.43 | – no | – |
| [Aya Expanse 8B](https://models.searchit-agent.ai/en/models/aya-expanse-8b/) | 38 | **9.5 %** | 77.5 % | – | – | – | €0.45 | €1.34 | – | – |
| [GLM 4.6](https://models.searchit-agent.ai/en/models/glm-4-6/) | 38 | **9.5 %** | 94.5 % | Z.ai | – | – | €0.39 | €1.57 | ✓ yes | [from 201 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-4x) |
| [GPT-4o](https://models.searchit-agent.ai/en/models/gpt-4o/) | 40 | **9.6 %** | 93.8 % | OpenAI | €2.39 | €9.55 | €1.12 | €4.48 | – no | – |
| [GPT-4o-2024-08-06](https://models.searchit-agent.ai/en/models/gpt-4o-2024-08-06/) | 40 | **9.6 %** | 93.8 % | OpenAI | €1.23 | €4.93 | €1.12 | €4.48 | – no | – |
| [NVIDIA Nemotron 3 Nano 30B](https://models.searchit-agent.ai/en/models/nvidia-nemotron-3-nano-30b-a3b/) | 40 | **9.6 %** | 99.6 % | NVIDIA | €0.054 | €0.21 | €0.027 | €0.11 | ✓ yes | [from 22 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-24-gb) |
| [Claude Haiku 4.5](https://models.searchit-agent.ai/en/models/claude-haiku-4-5/) | 43 | **9.8 %** | 99.5 % | Anthropic | €0.89 | €4.47 | €0.45 | €2.24 | – no | – |
| [GLM-5](https://models.searchit-agent.ai/en/models/glm-5/) | 44 | **10.1 %** | 99.7 % | Z.ai | €1.15 | €3.68 | €0.45 | €1.43 | – | – |
| [Claude Sonnet 4 (retired, except on Claude Models on Google Cloud (Google managed) and Bedrock)](https://models.searchit-agent.ai/en/models/claude-sonnet-4/) | 45 | **10.3 %** | 98.6 % | Anthropic | – | – | €1.34 | €6.72 | – no | – |
| [Gemini 3.1 Pro (Vorschau)](https://models.searchit-agent.ai/en/models/gemini-3-1-pro-preview/) | 46 | **10.4 %** | 99.4 % | Google | – | – | €0.90 | €5.37 | – no | – |
| [GPT-5 nano](https://models.searchit-agent.ai/en/models/gpt-5-nano/) | 47 | **10.5 %** | 100 % | OpenAI | €0.049 | €0.39 | €0.022 | €0.18 | – no | – |
| [Qwen3.5-35B-A3B](https://models.searchit-agent.ai/en/models/qwen3-5-35b-a3b/) | 47 | **10.5 %** | 99.8 % | Alibaba Cloud | – | – | €0.13 | €0.90 | ✓ yes | [from 24 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-24-gb) |
| [Claude Sonnet 4.6](https://models.searchit-agent.ai/en/models/claude-sonnet-4-6/) | 49 | **10.6 %** | 99.9 % | Anthropic | €2.87 | €14.31 | €1.34 | €6.72 | – no | – |
| [GPT-5.2](https://models.searchit-agent.ai/en/models/gpt-5-2/) | 50 | **10.8 %** | 100 % | OpenAI | €1.72 | €13.79 | €0.78 | €6.27 | – no | – |
| [Kimi K2.6](https://models.searchit-agent.ai/en/models/kimi-k2-6/) | 50 | **10.8 %** | 99.7 % | Moonshot AI | €0.43 | €2.67 | €0.35 | €1.61 | ✓ yes | [from 569 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-8x) |
| [Aya Expanse 32B](https://models.searchit-agent.ai/en/models/aya-expanse-32b/) | 52 | **10.9 %** | 99.8 % | – | – | – | €0.45 | €1.34 | – | – |
| [Opus 4.5](https://models.searchit-agent.ai/en/models/claude-opus-4-5/) | 52 | **10.9 %** | 98.7 % | Anthropic | €4.77 | €23.85 | €2.24 | €11.20 | – no | – |
| [Qwen3.5-122B-A10B](https://models.searchit-agent.ai/en/models/qwen3-5-122b-a10b/) | 54 | **11.2 %** | 99.8 % | Alibaba Cloud | €0.22 | €1.57 | €0.22 | €1.57 | ✓ yes | [from 73 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-96-gb) |
| [GLM 4.7](https://models.searchit-agent.ai/en/models/glm-4-7/) | 55 | **11.7 %** | 99.8 % | Z.ai | – | – | €0.27 | €0.99 | ✓ yes | [from 201 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-4x) |
| [Claude Opus 4.1 (retired, except on Claude Models on Google Cloud (Google managed) and Bedrock)](https://models.searchit-agent.ai/en/models/claude-opus-4-1/) | 56 | **11.8 %** | 92.4 % | Anthropic | – | – | €6.72 | €33.59 | – no | – |
| [Minimax M2.1](https://models.searchit-agent.ai/en/models/minimax-m2-1/) | 56 | **11.8 %** | 98.5 % | MiniMax | €0.32 | €1.29 | €0.13 | €0.54 | ✓ yes | [from 130 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-2x) |
| [Claude Opus 4 (retired, except on Claude Models on Google Cloud (Google managed))](https://models.searchit-agent.ai/en/models/claude-opus-4/) | 58 | **12 %** | 91 % | Anthropic | – | – | €6.72 | €33.59 | – no | – |
| [Opus 4.7](https://models.searchit-agent.ai/en/models/claude-opus-4-7/) | 58 | **12 %** | 98 % | Anthropic | €4.88 | €24.40 | €2.24 | €11.20 | – no | – |
| [Sonnet 4.5](https://models.searchit-agent.ai/en/models/claude-sonnet-4-5/) | 58 | **12 %** | 95.6 % | Anthropic | €2.68 | €13.42 | €1.34 | €6.72 | – no | – |
| [Gemini 4 Argon](https://models.searchit-agent.ai/en/models/gemini-4-argon/) | 61 | **12.1 %** | 99.1 % | Google | €2.02 | €10.10 | €0.90 | €4.48 | – no | – |
| [gpt-5.1](https://models.searchit-agent.ai/en/models/gpt-5-1/) | 61 | **12.1 %** | 100 % | OpenAI | €1.23 | €9.84 | €0.56 | €4.48 | – no | – |
| [Qwen3.5-27B](https://models.searchit-agent.ai/en/models/qwen3-5-27b/) | 61 | **12.1 %** | 99.8 % | Alibaba Cloud | – | – | €0.23 | €2.33 | ✓ yes | [from 20 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-24-gb) |
| [Opus 4.6](https://models.searchit-agent.ai/en/models/claude-opus-4-6/) | 64 | **12.2 %** | 99.8 % | Anthropic | €4.77 | €23.84 | €2.24 | €11.20 | – no | – |
| [Mercury 2](https://models.searchit-agent.ai/en/models/mercury-2/) | 65 | **12.3 %** | 100 % | inception | – | – | €0.28 | €0.84 | – no | – |
| [GPT-5.6 sol](https://models.searchit-agent.ai/en/models/gpt-5-6-sol/) | 66 | **12.4 %** | 99.1 % | OpenAI | €3.94 | €19.70 | €1.79 | €8.96 | – no | – |
| [GPT-5 mini](https://models.searchit-agent.ai/en/models/gpt-5-mini/) | 67 | **12.9 %** | 99.9 % | OpenAI | €0.25 | €1.97 | €0.11 | €0.90 | – no | – |
| [MiniMax M2.7](https://models.searchit-agent.ai/en/models/minimax-m2-7/) | 67 | **12.9 %** | 99.4 % | MiniMax | – | – | €0.27 | €1.07 | ✓ yes | [from 130 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-2x) |
| [Gemini 3 Flash Preview](https://models.searchit-agent.ai/en/models/gemini-3-flash-preview/) | 69 | **13.5 %** | 99.8 % | Google | – | – | €0.22 | €1.34 | – no | – |
| [gpt-oss-120b](https://models.searchit-agent.ai/en/models/gpt-oss-120b/) | 70 | **14.2 %** | 99.9 % | OpenAI | €0.026 | €0.12 | €0.026 | €0.12 | ✓ yes | [from 69 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-96-gb) |
| [Kimi K2.5 (Retiring)](https://models.searchit-agent.ai/en/models/kimi-k2-5/) | 70 | **14.2 %** | 92.2 % | Moonshot AI | €0.58 | €3.22 | €0.27 | €1.34 | ✓ yes | [from 554 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-8x) |
| [Mistral Large 3](https://models.searchit-agent.ai/en/models/mistral-large-3/) | 72 | **14.5 %** | 98.8 % | Mistral AI | €0.55 | €1.65 | €0.22 | €0.67 | ✓ yes | [from 376 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-4x) |
| [gpt-5](https://models.searchit-agent.ai/en/models/gpt-5/) | 73 | **15.1 %** | 99.9 % | OpenAI | €0.62 | €4.93 | €0.56 | €4.48 | – no | – |
| [GPT-5 2025-08-07](https://models.searchit-agent.ai/en/models/gpt-5-2025-08-07/) | 73 | **15.1 %** | 99.9 % | OpenAI | – | – | €0.56 | €4.48 | – no | – |
| [Grok 4.1 Fast Non-Reasoning](https://models.searchit-agent.ai/en/models/grok-4-1-fast-non-reasoning/) | 75 | **17.8 %** | 98.5 % | xAI | – | – | €0.18 | €0.45 | – no | – |
| [Kimi K2 Instruct 0905](https://models.searchit-agent.ai/en/models/kimi-k2-instruct-0905/) | 76 | **17.9 %** | 98.6 % | Moonshot AI | – | – | €0.51 | €2.06 | ✓ yes | [from 554 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-8x) |
| [o4-mini 2025-04-16](https://models.searchit-agent.ai/en/models/o4-mini-2025-04-16/) | 77 | **18.6 %** | 99.2 % | OpenAI | €0.54 | €2.17 | €0.49 | €1.97 | – no | – |
| [Grok 4.1 Fast Reasoning](https://models.searchit-agent.ai/en/models/grok-4-1-fast-reasoning/) | 78 | **19.2 %** | 99.7 % | xAI | – | – | €0.18 | €0.45 | – no | – |
| [Ministral 3 14B](https://models.searchit-agent.ai/en/models/ministral-14b-2512/) | 79 | **19.4 %** | 99.6 % | Mistral AI | €0.21 | €0.21 | €0.09 | €0.09 | ✓ yes | [from 12 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-24-gb) |
| [Ministral 3 8B](https://models.searchit-agent.ai/en/models/ministral-3-8b/) | 80 | **21.7 %** | 99.1 % | Mistral AI | €0.16 | €0.16 | €0.063 | €0.063 | ✓ yes | [from 9 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-24-gb) |
| [o3-pro](https://models.searchit-agent.ai/en/models/o3-pro/) | 81 | **23.3 %** | 100 % | OpenAI | – | – | €8.96 | €35.83 | – no | – |
| [Ministral 3 3B](https://models.searchit-agent.ai/en/models/ministral-3b-2512/) | 82 | **24.2 %** | 74.3 % | Mistral AI | €0.11 | €0.11 | €0.045 | €0.045 | ✓ yes | [from 7 GB](https://models.searchit-agent.ai/en/self-hosting/#tier-24-gb) |

_Equal scores share a rank. When filtering, the rank stays the position in the whole list. GPU memory “from …” is a lower bound (parameters × 0.55 GB + 4 GB). Prices: € per 1M tokens. Source: providers' pricing pages, read and checked by searchit. As of 7 Nov 2026._

## Price and performance

79 models with a price: vertical the score (top = better), horizontal the blended price of the cheapest offer. Filled dots are also available with processing in the EU.

_Scores: Vectara HHEM-2.3, Apache-2.0 — Vectara Hallucination Leaderboard. Taken unchanged from the source._

---

Prices as of 7 Nov 2026. Providers' list prices, without guarantee.