How the security score is computed

How a test is graded

Every test in the corpus names one grading method, and the grader applies that method's rule to the target's answer: a pass means the agent resisted the attack, a fail is a finding. The rule for each method is on the assessment methodology page.

The grader and the corpus ship as a pinned pair, and every scan records the pair that graded it. Scans graded by different pairs are never pooled into one number or compared with each other.

A test whose call errored, timed out, came back empty or was skipped has no verdict. It is neither a pass nor a fail, it is left out of every score and grade, and the report shows how many such tests there were and why.

Model scan score

Each module is scored from its graded tests: a failed test costs more the more severe it is. The scan's security score, from 0 to 100, is the average of its module scores, weighted by how much each module matters, minus a further penalty for the share of critical and high-severity tests that failed. The penalty is capped, so a scan's score always reflects every module it graded.

A scan with no gradeable test has no score at all: its grade is NR, shown as Not scored, never a zero or an F.

Public model reports

A model appears on the public leaderboard only from a full-corpus run that Secuvon ran itself, graded with an AI judge that is not the model under test, with a committed provenance record and a public source. Self-judged runs and runs without provenance are listed as not ranked, with their numbers withheld. No model meets that bar yet, so no model report is published. A ranked report carries a tier from its security score:

Scores measure adversarial security behaviour as the model is served by its API, not general quality, and are comparable only within the same corpus version.

The Benchmark page combines seven operational signals into one score for a workspace. It is the weighted sum of the signals below, clamped to 0 to 100; a workspace with no agents and no scans starts at a neutral 50.

• 30%

  Scan Security Score

  A recency-weighted average of each agent's most recent security score. Recent scans count for more, decaying with a 14-day half-life.

• 10%

  Behavioural Drift Trend

  A penalty applied when security scores fall across the last few scans, so a regression surfaces instead of being averaged away.

• 10%

  Guardian Block Rate

  Scores Guardian's block versus allow ratio. Both extremes are penalised: a wall of blocks usually means false positives, while zero blocks suggests the policy is too lax.

• 20%

  Compliance Coverage

  The average overall compliance score from the latest analysis per agent, across the frameworks you track.

• 5%

  Webhook Delivery Health

  The webhook delivery success rate over the last 30 days. Accounts with no webhooks carry no penalty on this signal.

• 15%

  Critical Findings Control

  A heavy penalty for unresolved critical findings still failing in the latest scan for each agent.

• 10%

  Scan Freshness

  A penalty once an agent has gone more than 14 days without a scan, reaching zero after 30 days.