Many SWE-bench-Passing PRs Would Not Be Merged into Main
benchmarksMETR BlogMar 10, 2026
Summary: We find that roughly half of test-passing SWE-bench Verified PRs written by mid-2024 to mid/late-2025 agents would not be merged into main by repo maintainers, even after adjusting for noise in maintainer merge decisions. Since the agents are not g...
Impact: MediumConfidence: MediumNovelty: Material
benchmarkssource-feed-1bdee3f1-6eca-4e71-a9df-4f6430ba5051agentscodingopen-sourcesafety