Verification

After this lesson, the reader should understand:

• 01Define the minimum verifiable record of a decision.
• 02Distinguish verification from audit and from explanation.
• 03Design a verification path for an external regulator.

A regulator re-runs a year of decisions

What would a regulator need in order to re-execute last year's decisions without your systems?

The artifact versions in force at each point in time, the recorded facts, and a conforming runtime. Nothing about your infrastructure, your model, or your staff. If re-execution requires your cooperation to interpret anything, the record is an explanation rather than a verification^[1] — and explanations do not survive disputes. The NOMOS reference implementation makes this checkable now. Every keyless query against a published authority mints a permanent, sealed transcript at its own URL, carrying the artifact id and version, the seal hash, the facts supplied, the rule the verdict turned on and its source citation, and an audit hash. Anyone can re-fetch that transcript and, with the published public key, verify the seal offline — no account, no access to the system that produced it. The compiled EU AI Act artifact is one live instance: the transcript at nomosprotocol.com/queries/qry_a5e0603057d9b0c4373f is re-servable and resolves to the same verdict every time. The strongest objection is that re-execution verifies the arithmetic, not the truth. If the recorded facts were wrong, re-running them reproduces the wrong decision perfectly: verified garbage. Concede it, and be exact about what a verification establishes. It shows that this outcome follows from this artifact applied to these facts — nothing about whether the facts were true. The reference implementation illustrates the gap honestly: its keyless query path records the facts exactly as the caller supplied them, so a caller who misstates a fact receives a genuine, permanent transcript of a verdict on the misstatement. Whether the facts were true is a separate question with separate machinery — provenance for each value, attestation by a party entitled to vouch for it, or a check against a source the caller does not control. Where the reference implementation spends against a real budget, for example, it refuses to honour a decision whose recorded balance does not match the ledger's actual one. Verification of evaluation and verification of fact are different layers, and a record should say which it offers. The limit is privacy. A record complete enough for a stranger to re-execute contains the facts, and the facts are often personal data. Publishing them conflicts with data-protection law's principle of minimisation^[2]; withholding them makes the record unverifiable by outsiders. There are partial technical answers — committing to facts with a hash so they can be revealed selectively^[3], or, at the frontier, proofs that a computation was performed correctly without disclosing its inputs^[4] — but the default design choice is a real trade-off, not a solved problem. Who may verify which decisions, and with how much of the underlying data, is a question for the institution's governance before it is one for its engineers.

• Should verification be open to the decision subject as well as the regulator?
• What is disclosed and what is withheld in a verification record?
• Who certifies a conforming runtime?
• A verification shows the outcome followed from the recorded facts. What would it take to verify the facts themselves?
• How much of a decision's underlying data should a verifier see, and who decides?

No responses yet. Be the first to answer one of the questions above.

Construct a verification record for one decision.

1. 01Assemble artifact version, facts, trace, and outcome.
2. 02Remove anything that depends on your systems to interpret.
3. 03Hand it to a colleague and ask them to reach the same outcome.
4. 04Record what they had to ask you.
5. 05Mark each fact by how its truth could be checked: attested, cross-checked against a source, or taken on the caller's word.