Article tracking: Trump
Determine whether an article is describing a new action/policy by the Trump administration.
| Model | Pass rate | Tests | Avg tokens |
|---|---|---|---|
| claude-opus-4-0 | 200 | – | |
| claude-haiku-4-5 | 200 | 31 | |
| kimi-k2-thinking (moonshotai) | 200 | 330 | |
| gpt-4o-mini | 200 | 2 | |
| qwen3-235b-a22b-2507 | 200 | – | |
| qwen3.6-35b-a3b (venice/fp8) | 200 | 683 | |
| deepseek-chat-v3-0324 | 200 | – | |
| qwen3-14b | 200 | – | |
| claude-3-5-sonnet-20241022 | 200 | 23 | |
| qwen3-30b-a3b-2507 | 200 | 2 | |
| qwen3.8-27b (venice/fp8) | 200 | 174 | |
| claude-sonnet-4-0 | 200 | – | |
| claude-sonnet-5 | 200 | 5 | |
| gpt-oss-20b | 199 | – | |
| claude-fable-5-1 | 200 | 48 | |
| claude-opus-5 | 200 | 74 | |
| gemma-3-12b | 200 | – | |
| gpt-oss-120b | 200 | 119 | |
| gpt-5.5 | 21 | – | |
| gpt-5.6-luna | 19 | – | |
| gpt-6-astra | 23 | – |
Full results
Loading results…