AI Safety Scores Mostly Measure How Big the Model Is

Wed, 26 Aug 2026 00:50:49 +0000

A 2024 analysis found that scores on popular AI safety benchmarks correlate so strongly with general model capability that you can largely predict a model’s “safety” rating just by knowing how well it does on MMLU. No actual safety research required. Just make the model smarter, and the safety score follows automatically, like a shadow.

I’ve been thinking about what that means. On its face, it sounds like good news — bigger models are safer models. But sit with it for a second, and something starts to feel wrong.

Goodharts-Law on BRYSGO

AI Safety Scores Mostly Measure How Big the Model Is