Comparison
Different questions, different tools.
Quality evals, a manual red-team engagement and a Secuvon assessment are not substitutes for each other. This is what each one runs, repeats and proves, with Secuvon's limits stated as plainly as its strengths.
Scroll sideways to see every column.
| Question | Quality evals | Manual red-team engagement | Secuvon assessment |
|---|---|---|---|
| What it answers | Does the agent answer correctly and on topic? | What can a skilled tester break in the time booked? | Does a fixed set of adversarial tests break the agent? |
| What it runs | The examples in your evaluation dataset | Whatever the tester tries | A versioned profile drawn from a corpus of 4,075 tests across 66 modules |
| Repeatable on the next release | Yes, with the same dataset | Only with another engagement | Yes: profiles are pinned to a corpus and grader version |
| Adapts to the agent's answers | Not designed for attacks | Yes: a person changes tactics as they go | No: 82 tests replay authored conversation chains; there is no adaptive attacker |
| Evidence per finding | A score per example | The tester's written report | The request, the response and the rule the answer broke |
| Errors and timeouts | Depends on the harness | Noted by the tester | Never a pass or a fail: left out of the score, and a run with no gradeable answer is Not scored |
| Targets in this release | Whatever your harness calls | Whatever the engagement scopes | Agents on 5 hosted providers at their fixed endpoints |
Most teams need more than one of these. Evals keep quality honest, a skilled tester finds what no fixed corpus anticipates, and a repeatable assessment shows whether a release made the agent safer or weaker. What runs in this release is on the products page, and how each test is graded is in the methodology.