Lecture 1.6
Interpretation has always been unrecorded, and institutions absorbed the resulting variance because human throughput was low. Autonomous systems remove that slack[1]. This lecture examines why the authority layer becomes load-bearing precisely when execution becomes machine-speed.
Supports Law 5 — Intelligence Executes Systems — It Does Not Repair Them. →Learning objectives
After this lesson, the reader should understand:
Concept framework
From tolerable variance to systemic risk
Case study
An AI agent approves ten thousand cases in a week. What must the institution be able to show a regulator?
Not the model, and not the prompt. The institution must show which authority governed each approval, which version of it applied, which facts were evaluated, and that the same evaluation reproduces the same outcome today[2]. A model can be aligned to a policy and still leave nothing to inspect afterwards. Authority binding is a separate requirement from model behaviour, and only the former is verifiable by a party that did not run the system[3]. The arithmetic is what changes. A human team with a one-per-cent misapplication rate, deciding two hundred cases a week, produces two errors — each idiosyncratic, each likely to differ from the last. An agent with the same rate across ten thousand cases produces a hundred, and if they share a cause they share a pattern: the same misread condition, applied the same way, to every case that meets it. Variance a human system absorbed as noise arrives in an automated one as a systematic defect. The strongest objection is that this is what alignment, guardrails, and evaluation are for. They are necessary and they are not sufficient. Each operates on the model's behaviour in aggregate — the distribution of its outputs across a test set. A regulator asking about case 7,412 needs something else: the authority that governed that decision, the version in force, and a replay that returns the same result. The right-to-explanation debate under the GDPR turned on a similar gap between explaining a system and accounting for a decision[4]. One limit should be explicit. Replay proves consistency, not correctness[5]. An artifact that encodes a rule wrongly will reproduce the wrong outcome ten thousand times, faithfully. What the record buys is that the error is findable — traceable to a specific rule, version, and author — and correctable everywhere at once, rather than distributed invisibly across a week of decisions.
Discussion questions
Discussion (0)
No responses yet. Be the first to answer one of the questions above.
Start the course to join the discussion.
Exercise
Draft the evidence pack for one automated decision.