Capability Claims
In plain words: Expiring capability claims
How healthy evaluation results become a scoped, versioned, expiring claim about what a specific model or setup can do, and how routing, spending or training owners may consult it.
- 1A claim pins exact model, adapter, tools, config, cases and environment, plus sample size, uncertainty, cost and latency.
- 2Accuracy, latency, VRAM, cost and behaviour stay separate axes; no single score.
- 3Any change needs new results; today the Traffic Controller uses a fixed readiness order, not quality or price.
Designed No claim or evaluation-driven routing path runs yet.
Words on this page, decoded click one for its card
Two Outcomes report successful work and the same favorable verdict. One came from a constrained local Environment with a large sample, low cost, and an expiry next month; the other used different tools, fewer Cases, higher cost, and a shorter evidence window. Their shared verdict—or one impressive number—cannot choose a universal winner. The useful result is an exact statement of what each subject revision demonstrated under named conditions.
Capability claims are Designed; no claim or routing path runs. State of Work records maturity; ADR 34 owns the claim law.
Shape the supported claim
A claim derives only from a healthy, version-pinned TrialSuite@1 and its retained Outcomes. This
page defines no health algorithm. The evidence must survive its declared controls, harness and
Evaluator health, leakage checks, calibration review, and required receipts.
Make the claim precise enough for a consumer to decide whether its evidence applies. Pin the exact AnimatorModel Runtime, model, adapter, toolset, and configuration revisions; name the task class, Cases, Rubric, Evaluators, and Environment. Retain sample count, controls, verdict distribution, uncertainty, and noise, so a favorable verdict does not hide how often it held. Record cost and latency per admitted success, counting failed attempts too. Creation time, expiry, and evidence references delimit how long and from what material the claim draws support.
Accuracy, latency, VRAM, cost, restraint, and tool behavior remain separate axes. The consumer decides which dimensions matter for its present request. DriftEvaluator does not collapse them into one scalar.
Expire without rewriting history
A claim can guide current decisions only while its evidence is present, healthy, and unexpired. Changing the subject, toolset, configuration, Rubric, Evaluator, or Environment requires new Outcomes and a new claim. A silent refresh would borrow a conclusion from conditions no longer being tested. Keep the stale claim as historical evidence, without allowing it to steer current decisions.
Later loss, leakage, changes in Evaluator behavior, or contradiction invalidates only claims whose complete support closure has broken. Independently supported claims remain intact. Recovery means a new bounded trial and a new claim, while causal uncertainty can move through Returning findings.
Missing or stale evidence leaves the DispatcherTraffic Controller's existing deterministic readiness order in place. It neither introduces retry through another candidate nor creates an intelligence floor for untested subjects.
Let each consumer decide
A fresh admitted claim may inform Dispatcher only after WardSecurity Desk, compatibility, availability, privacy, and authority have produced an eligible set. DispatcherTraffic Controller retains selection and lease ownership. Its current readiness order prefers open admission, then non-error state, then active over inactive and warm over merely active, with stable name and capability key tie-breakers; it does not optimize quality or price. No evaluation-driven ordering is delivered.
For Toll, retained cost and latency evidence may inform spending policy. It neither authorizes payment nor supplies a budget, signature, or settlement. A cheaper local success can be relevant without making all PortalCloud Model use wasteful.
SoulforgeModel Trainer may compare baseline evidence, sealed-holdout Outcomes, expected change, and regressions for a frozen candidate. Passing DriftEvaluator establishes eligibility only. The candidate handoff keeps that eligibility separate from the externally owned Promotion Decision; a claim cannot admit corpus, register a candidate, promote, route, or grant capability.
A capability claim remains what its evidence can carry: versioned, scoped, expiring, and useful only inside the consuming owner's decision.