FrontierCode基准测试揭示AI编码新纪元FrontierCode Benchmark Ushers in New Era of AI Coding
Cognition Labs的Swyx强调了METR_Evals的FrontierCode基准测试,显示SWE-Bench一半以上的结果是无法合并的废料。FrontierCode包含3000多个用于可维护代码质量的评判标准,Opus 4.8在Diamond级别仅得13.8%。它标志着从自动完成(HumanEval)和通过测试(SWE-Bench)转向‘可维护代码’时代。2025年底的快速进步显示模型正在饱和较简单的任务,从而启用更高级的agentic编码循环。该基准将每年演进以匹配AI能力增长。
Swyx from Cognition Labs highlights METR_Evals' FrontierCode benchmark, showing over half of SWE-Bench results are unmergeable slop. FrontierCode features 3000+ rubrics for maintainable code quality, with Opus 4.8 scoring just 13.8% on Diamond tier. It marks the shift to 'Maintainable Code' era after autocomplete (HumanEval) and test-passing (SWE-Bench). Rapid progress in late 2025 shows models saturating easier tasks, enabling advanced agentic coding loops. The benchmark will evolve annually as AI capabilities grow.
查看原文 →