Evidence and Admissibility

After this lesson, the reader should understand:

• 01State the properties that determine whether a decision log can support inference.
• 02Explain why a log with no outcome variety carries no information about a boundary.
• 03Decide, before analysing, whether a dataset is admissible.

The log that proves nothing

An institution supplies 4,000 approval decisions. Every one was approved. What can be recovered from it?

Nothing about the boundary. A boundary is the place where the answer changes, and this record contains no change. Stated formally, the outcome distribution carries zero entropy^[1]: knowing the input tells you nothing you did not already know, because the output never varies. The dataset is large, clean, complete, and useless for this purpose^[2] — which is why volume alone is a poor admissibility test, and why the four tests must be applied together. The correct response is not to analyse it more cleverly. It is to declare the evidence insufficient, and say precisely why. The NOMOS reference implementation encodes exactly this. Its triage gate scores a decision log on the same four axes — decision volume, outcome variety, field completeness, temporal coverage — and weights them together rather than letting any one carry the verdict. Outcome variety is the Shannon entropy of the outcome distribution normalised against a balanced split, not a count of distinct values: a log of 49,998 approvals and two refusals still scores near zero. A single outcome class scores zero outright and the log is rejected with a message that names the reason — without refusals there is nothing against which to locate the policy line. Reconstruction never runs on evidence that has not cleared the gate. The strongest objection is that rejection is paternalistic. Any data is better than none, a skilled analyst can still learn something from 4,000 approvals, and an institution that has paid to extract its records deserves more than a refusal. The objection mistakes what is being refused. Admissibility is relative to a claim, not a property of the dataset^[3]. The same log is perfectly admissible for other questions — how many requests arrive, how fast they are processed, what kinds of supplier appear. It is inadmissible only for the one claim that needs both sides of a line: where the line is. Declaring it insufficient for that claim, while saying what it can support, is the most useful thing an analyst can tell the institution. Passing the four tests is necessary, and still not sufficient. A log can have volume, variety, completeness, and coverage and yet describe the wrong population. If applicants are informally discouraged before they file — told by a colleague not to bother — the refusals that would have defined the boundary never reach the record at all. The log then records only the decisions the institution allowed to become decisions. Statisticians call this selection^[4], and its close relative is data missing for reasons connected to the outcome^[5]; neither is visible from inside the log. The admissibility gate checks what the record contains. Asking how the record came to contain it is a separate question, and it has to be asked of the institution, not of the data.

• Is a rejected dataset a failure of the institution, or of the request made of it?
• What should a regulator require before accepting an inferred boundary?
• How should an institution begin recording decisions it does not currently record?
• Does refusing to analyse inadmissible evidence protect the institution, or obstruct it?
• What decisions might your institution make before anything reaches the record — and what would their absence do to a reconstructed boundary?

No responses yet. Be the first to answer one of the questions above.

Test a real decision log for admissibility.

1. 01Count the decisions, and the distinct outcomes.
2. 02Compute the proportion falling in each outcome class.
3. 03List the policy-relevant fields and measure their completeness.
4. 04State the period covered, and the decision cadence within it.
5. 05Write one sentence: admissible, admissible with reservations, or inadmissible — and why.
6. 06Name one claim the log could support even if it is inadmissible for locating a boundary.
7. 07Describe how a case reaches the log, and where a case could be filtered out before it does.