Is the latest hot benchmark again just slop like the other benchmarks? 😞

Can you re-run under Hermes Agent with Qwen3.6? Give it grounding in POSIX specs, RFCs ..., llvm source, lean4 source, linux source as local directories. I think it might go a heck of a lot higher…

DeepSWE reported DeepSeek as expensive and weak: ~8% solve rate, $4.22/task. someone audited it, turns out it billed cache hits (78% of tokens) at the full price, no discount. real cost: ~$0.30/task.…