Module 3 · The Observed System

Lecture 3.3

Evidence and Admissibility

Reconstruction is only as sound as the record it rests on. This lecture treats a decision log as evidence subject to admissibility tests, and establishes the conditions under which a body of decisions can support any claim about a boundary at all.

Supports Law 6Every Automated Decision Must Be Independently Verifiable.

Learning objectives

After this lesson, the reader should understand:

  • 01State the properties that determine whether a decision log can support inference.
  • 02Explain why a log with no outcome variety carries no information about a boundary.
  • 03Decide, before analysing, whether a dataset is admissible.

Concept framework

Four admissibility tests

  1. 01Volume — enough decisions to separate a boundary from a coincidence
  2. 02Outcome variety — both sides of the boundary actually observed
  3. 03Completeness — the fields the policy depends on are populated
  4. 04Temporal coverage — a period long enough to reflect the decision cadence

Case study

The log that proves nothing

An institution supplies 4,000 approval decisions. Every one was approved. What can be recovered from it?

Nothing about the boundary. A boundary is the place where the answer changes, and this record contains no change. Stated formally, the outcome distribution carries zero entropy: knowing the input tells you nothing you did not already know, because the output never varies. The dataset is large, clean, complete, and useless for this purpose — which is why volume alone is a poor admissibility test, and why the four tests must be applied together. The correct response is not to analyse it more cleverly. It is to declare the evidence insufficient, and say precisely why.

Discussion questions

  • Is a rejected dataset a failure of the institution, or of the request made of it?
  • What should a regulator require before accepting an inferred boundary?
  • How should an institution begin recording decisions it does not currently record?
  • Does refusing to analyse inadmissible evidence protect the institution, or obstruct it?

Exercise

Test a real decision log for admissibility.

  1. 01Count the decisions, and the distinct outcomes.
  2. 02Compute the proportion falling in each outcome class.
  3. 03List the policy-relevant fields and measure their completeness.
  4. 04State the period covered, and the decision cadence within it.
  5. 05Write one sentence: admissible, admissible with reservations, or inadmissible — and why.

Research notes

  • Information theory — entropy as a measure of outcome variety (Shannon).
  • Statistical power — sample size and minimum detectable effect.
  • Evidence law — admissibility as a precondition, distinct from weight.
  • Data quality frameworks — completeness, timeliness, validity.