0
CWE-bench v2: More coverage, quality, and transparency
https://huggingface.co/blog/cwe-bench(huggingface.co)CWE-bench v2 is a benchmark designed to test the cybersecurity capabilities of AI models by tasking them with finding and fixing security weaknesses in real open-source code. This version introduces a public set of 100 tasks covering 91 different vulnerability classes, with fully programmatic and reproducible scoring to ensure fairness. The benchmark includes a public leaderboard that tracks the performance of both open-weight and API-only models like Gemini, Claude, and GPT. Current results show Gemini 4 Argon leading, but even the top models fail on a significant portion of the tasks, indicating room for improvement. The entire benchmark can be run directly from the Hugging Face Hub and can also be used to train reinforcement learning agents for defensive purposes.
0 points•by ogg•1 hour ago
Comments (0)
No comments yet. Be the first to comment!
Have an account? Log in to join the discussion.