Political fundraising emails
A random sample of 100 fundraising emails from Derek Willis.
| Model | Pass rate | Tests | Avg tokens |
|---|---|---|---|
| gemini-1.5-flash-002 | 100 | – | |
| gemma-3-12b | 100 | – | |
| gpt-5-mini-2025-08-07 | 100 | – | |
| gpt-4o-mini | 100 | – | |
| o4-mini-2025-04-16 | 100 | – | |
| qwen3-14b | 100 | – | |
| gemini-2.0-flash-lite | 100 | – | |
| gpt-5-nano-2025-08-07 | 100 | – | |
| gpt-6-astra | 100 | 9 | |
| claude-3-5-sonnet-20241022 | 100 | 9 | |
| claude-fable-5-1 | 100 | 13 | |
| claude-opus-5 | 100 | 37 | |
| gpt-4.1-mini-2025-04-14 | 100 | – | |
| gpt-oss-120b | 100 | – | |
| claude-haiku-4-5 | 100 | 9 | |
| gpt-oss-20b | 100 | – | |
| qwen3-30b-a3b-2507 | 100 | 6 | |
| kimi-k2-thinking (moonshotai) | 100 | 557 | |
| gemini-2.0-flash | 100 | – | |
| qwen3.8-27b (venice/fp8) | 100 | 406 | |
| qwen3.6-35b-a3b (venice/fp8) | 100 | 1.2k | |
| gpt-5.5 | 5 | – | |
| gpt-5.6-luna | 19 | – |
Full results
Loading results…