Adjudication

After this lesson, the reader should understand:

• 01Separate detecting a divergence from judging it.
• 02Explain why outcome evidence, not the divergence itself, determines harm.
• 03Apply a defensible standard before asserting a causal claim — and recognise when to withhold one.

The tightening that saved nothing

Officers escalate at £48,000 against a documented £50,000. Is the drift harmful?

The divergence alone cannot answer it. Escalating earlier is more cautious, and caution is not self-evidently correct: it consumes review capacity, delays suppliers, and may deliver no measurable reduction in loss. The question is empirical — join the outcomes to the decisions, estimate the difference the tightening actually made, and adjust for the covariates that plausibly drive both the escalation and the outcome. Two disciplines apply. First, when many rules are examined at once, some will appear significant by chance alone, and the threshold must be corrected to account for that. Second, when the effect is small and the sample thin, the honest verdict is inconclusive — a category that must remain available, because a method that always returns an answer is not measuring anything.

• What standard of evidence should precede telling an institution its practice is harmful?
• Who decides which outcome counts as the bad one?
• Is a beneficial divergence a reason to amend the document?
• What is the cost of reporting inconclusive — and the cost of not having the option?

Set the standard before you look.

1. 01Take one divergence you have already found.
2. 02Name the outcome that would make it harmful, in the institution's own terms.
3. 03List the covariates that could produce a spurious association.
4. 04State, in writing and in advance, the evidence threshold you will accept.
5. 05Decide in advance what result would make you report inconclusive.

• Causal inference — average treatment effect, confounding, refutation (Pearl; Imbens & Rubin).
• Multiple comparisons — false discovery rate control (Benjamini & Hochberg).
• Statistical power — detectable effect size and required sample.
• Metrology — reporting uncertainty alongside measurement.