MLCommons announced the release of AILuminate Safety 1.2.1 and Jailbreak 1.1 versions. These tests are available in the new Candela service for comparative testing, which also features a user evaluator and percentile scoring.
According to MLCommons, the updates are designed to be resistant to benchmaxxing, cherry-picking and other manipulation methods. Safety scores are currently available, and Jailbreak results are expected later.
Practical significance depends on how detailed the methodology, attack sets and results will be published. For now the source provides only a brief product description and lacks independent verification or numerical outcomes.
editorial commentary
Why it matters
Possible consequence — the appearance of more comparable safety ratings of models, if the stated safeguards against manipulation are borne out in practice. The next observable signal will be the publication of Jailbreak results and methodological details. Significant uncertainty remains due to a single brief description without independent verification.