Show HN: Parsimony – an open-source benchmark for how much code AI models add
AI summary
Editorial policyParsimony is an open-source benchmark that measures how much redundant code AI models add by counting syntax elements, addressing the gap in existing benchmarks that ignore code bloat.
Key points
- •Existing benchmarks measure solve rate and token cost but do not penalize excess code that makes codebases harder to maintain long-term.
- •Parsimony uses patches from benchmarks like DeepSWE and SWE-bench Verified to count 'coding units' such as statements and calls.
- •It counts syntax elements rather than lines of code to prevent rewarding code cramming.