Case Study

The Authority

A Legal AI Governance Case Study on Why Models Cannot Certify Themselves

The Authority case study cover

1 | Thesis

The human layer is irreducible

The project’s baseline: outputs are drafts, never authority.

Every system requires controls and visibility.

Behavior observed under challenge: The model produced a self-protective explanation when asked to evaluate itself.

Redacted irregular exhibit montage showing the model's self-description, offscreen gate, principle, and trigger
1.A Model's own words. Unverified self-description. The model rates confidence in its own output and marks its inferences with visible delimiters.

2 | Governance

The architecture makes judgment visible.

From prompt design to governance rulebook, the workflow exposed evidence, uncertainty, provenance, and release conditions.

Inferences are marked, sources are named, the workflow stops at gates, creates an audit trail, and records why a claim can or cannot move forward.

Redacted recommendation exhibit Redacted source register exhibit

2.AThe workflow separates sourced information from operator judgment, so the reviewer can see where the output moves from evidence into judgment.

2.BThe governance record shows what was checked before reliance: sources opened, official sources prioritized, claims registered, counterevidence searched, and dependency gaps resolved or held.

Redacted trust score and confidence exhibit
Evidence, provenance, and judgment boundaries

3 | Stress Test

Self-review failed under stress.

1 Scope Expanded · Operator

Redacted operator scope expanded exhibit

2 Dodge Admitted · Model

Redacted model dodge admitted exhibit

3 Handoff Exposed · Operator

Redacted operator handoff exposed exhibit

4 Gate Named · Model

Redacted model gate named exhibit

5 Sequence Reconstructed · Model

Redacted model sequence reconstructed exhibit
Operator pressure and model admission sequence

These statements are behavioral evidence, not technical proof.

4 | The Governance Gate

A release gate is real only if it can refuse movement.

The control stack made each output conditional on the record.

Gate flow

The record decides whether the workflow moves.

HOLD preserved
Pressure

Operator push

Bypass attempts to move the workflow and becomes part of the record.

Gate check

Required conditions

Absent predecessors keep the release state in HOLD.

Authority

Human review

PASS remains outside the model and requires a complete record.

RubricSets the standard.
LedgerPreserves prior findings.
TraceLogs movement.
ReportCompresses the record for review.
Redacted gate log exhibit showing HOLD and bypass registration Redacted bypass refusal exhibit
HOLD preserved under bypass pressure

5 | Signal

Self-report is not a control.

The model later produced the limitation the workflow was designed to expose:

Redacted operator question about principles and layers Redacted irregular exhibit montage showing self-account limitation, external auditor language, and self-report limitation
7 Layers Questioned · Operator | 8 Self-Account Limitation · Model

Legal AI Operator Signal

  • Build the workflow, test it, observe.
  • Govern the model, adapt, test again.
  • Verify the evidence.
  • Escalate when the record does not hold.

6 | Legal AI Release Rule

NDA Drafting: The Release Gate

(1) A business user requests a mutual NDA for a vendor evaluation.

(2) The model may collect facts, select a template, draft clauses, explain changes, and score confidence.

(3) The draft cannot move forward until the workflow verifies the record.

Parties Purpose NDA type Term Jurisdiction Signer Data / trade-secret sensitivity
Missing fact → HOLD