What is the best harness for each model?
FrontierHarness Eval v1.1 results show that 64% of inference spend across 27,280 coding agent runs went to failed executions.
Key points
- •FrontierHarness Eval v1.1 evaluated 22 coding agent harnesses with 5 models on the same 62 tasks, 4 times each.
- •The best-value harness differed by model, with cost ranks shifting significantly between models.
- •With a fixed model, the most expensive harness cost 1.8 to 5.0 times as much per task as the cheapest.