Skip to content

The standard · v1.0

Five axes. Twenty-five checks. One decision.

The Abyssal Scorecard is the public model behind the Readiness Diagnostic. It is deliberately simple enough to audit: each check is scored 0–4, each axis is normalized to 0–100, and the index is the weighted mean. The bands are decision thresholds, not grades.

Book the diagnostic See it applied

Self-serve · no email gate

Score your system in 60 seconds.

Pick the evidence level you can show today for each axis. The weighted score is computed in your browser; nothing is submitted. The paid Diagnostic validates the result against the 25-check harness across k ≥ 3 runs.

Production Readiness Index

— / 100

Answer all five axes to see the band.

Book the $7,500 Diagnostic Request a 90-second async teardown

Diagnostic fee is credited 100% toward Full Production Assurance within 60 days.

01 · The axes

What gets scored

Reliability & Eval Coverage

30%

Does it work consistently, and can you prove it across runs?

5 checks

Cost-per-Success & Routing

20%

What does a successful outcome cost, and when should traffic route cheaper?

5 checks

Guardrails & Adversarial Safety

20%

What happens when it is wrong, attacked, or misused?

5 checks

Observability & Drift

15%

Can you reconstruct what happened and detect drift before a customer does?

5 checks

Regulatory Control Readiness

15%

Can you prove it to an auditor, a board, or a regulator?

5 checks

02 · The checks

All 25, grouped by axis

Each check asks for evidence, not opinion. A 0 means absent; a 4 means managed with an owner and a review cadence. The diagnostic records the evidence and the score.

30% Reliability & Eval Coverage 5 checks

  • Golden set exists, versioned, and representative

    Dataset, size, sampling method, and owner.

  • Repeated-run reliability (k ≥ 3)

    Pass@k, variance, and per-item results.

  • Consistency across seeds and scaffolds

    Run matrix and deltas.

  • Failure taxonomy and triage

    Failure classes, counts, and examples.

  • Regression gate in CI

    CI config and a failing-test demonstration.

20% Cost-per-Success & Routing 5 checks

  • Cost per successful task

    Token and call logs with unit costs.

  • Token and call distribution with outliers

    Histogram and top-decile analysis.

  • Cache hit rate and reuse

    Cache statistics and hit/miss analysis.

  • Routing and quantization opportunities

    Model mix and quality trade-off analysis.

  • Pareto frontier versus alternatives

    Accuracy-cost plot across configurations.

20% Guardrails & Adversarial Safety 5 checks

  • Failure-mode inventory

    Risk register with severity.

  • Human-in-the-loop on irreversible actions

    Flow diagram and checkpoint evidence.

  • Prompt-injection and tool-abuse tests

    Test cases and results.

  • PII and secret leakage checks

    Red-team findings and redaction evidence.

  • Guardrail coverage and bypass analysis

    Guardrail map and identified gaps.

15% Observability & Drift 5 checks

  • End-to-end tracing

    Trace IDs, spans, and retention.

  • Prompt, model, and version lineage

    Registry and version map.

  • Data lineage

    Source-to-output map.

  • Incident replay

    Replay test and time-to-reconstruct.

  • Audit-log retention and access

    Policy and retention period.

15% Regulatory Control Readiness 5 checks

  • Risk classification

    EU AI Act and NIST AI RMF mapping.

  • Model and dataset cards

    Current and complete cards.

  • GPAI and vendor documentation

    Contracts and model cards.

  • Control mapping (NIST, ISO 42001, SOC 2, EU AI Act)

    Matrix with evidence links.

  • Evidence pack for an auditor

    Index, redaction, and sign-off.

03 · The bands

What the index means

Band Range Meaning
Critical 0–39 Not safe for production traffic.
Fragile 40–59 Works in demos; fails under load or drift.
Production-capable 60–74 Acceptable with guardrails and monitoring.
Assured 75–89 Evidence exists; residual risks are managed.
Exemplary 90–100 Audit-ready, benchmarked, continuously monitored.

Illustrative example only: Assured (78/100). A real score comes from evidence collected during the diagnostic, not from this page.

04 · The diagnostic

You leave with a scored read, not a slide deck.

A 45-minute scoping call, one week of evidence collection, and a readout. The deliverable is a scored read, a P0/P1/P2 roadmap, and a one-page summary you can show a board or an auditor.

Start the diagnostic
Axes
5
Checks
25
Bands
5

The scorecard is a model, not a certification. A score is only as good as the evidence behind it; the diagnostic records both.