OpenAI says it found widespread task issues in SWE-Bench Pro, estimates ~30% of tasks are broken, and retracts its earlier recommendation to adopt the benchmark (OpenAI)
Culture Index
Score Breakdown
Relevance
3/25
Freshness
25/25
Authority
25/20
Brand Signal
8/15
Depth
4/15
5-Axis Cultural Radar
OpenAI : OpenAI says it found widespread task issues in SWE-Bench Pro, estimates ~30% of tasks are broken, and retracts its earlier recommendation to adopt the benchmark — Through a detailed audit, we find widespread task issues in SWE-Bench Pro and estimate that ~30% of the tasks are broken.
