Drift

In plain words: Evaluator overview

Designed 2 min read 6 glossary terms 13 connected pages
The short version written for the Snayu team

The planned evaluation area (Evaluator, or Drift) that turns a declared goal into test cases and rubrics, runs trials against an exact version, and returns findings that keep uncertainty, without authority to act on them.

Why read it: Read this before building any eval, benchmark or quality check, to understand the vocabulary and limits of evaluation findings.
  1. 1State the goal first as Cases and Rubrics; a trial is bound to the exact subject revision being tested.
  2. 2Execution status and verdict are separate; a blocked trial is not a failure.
  3. 3A good verdict grants no permission, routing, causal proof or promotion; owners decide what to do.

Designed No runnable evaluation path is delivered.

Words on this page, decoded click one for its card

Plug-in AreaWhat-if Simulat…Model TrainerEvaluator
Where it sits in LychD

Evaluator Drift

Tests whether something actually meets its goal. Its findings are evidence, never automatic approval.

It's like: Quality assurance with a lab notebook.

Click a dot to open that part's card.

Glossary page In the network

In this section 4 pages

Trials Evaluation trials and verdicts How an evaluation trial is pinned before it runs (cases, suite, rubric, evaluator, environment), how observations are recorded before judging, and the status and verdict vocabularies used in results. open → Capability Claims Expiring capability claims How healthy evaluation results become a scoped, versioned, expiring claim about what a specific model or setup can do, and how routing, spending or training owners may consult it. open → Returning Findings Tracing failures back to a cause How a failure seen downstream in a multi-app bundle (Composition Suite) is traced backward only as far as evidence supports, producing inert findings and correction requests instead of blame or reverse execution. open → Workflow Improvement Evaluating and improving workflows How to judge whether a workflow method is getting better, not just whether one output is right, by comparing whole runs on matched tasks, including cost, corrections and skipped checks, then returning a decision to the owner. open →
The original text Underlined words open their plain-English card.

By which star will you judge the wandering?

To judge a result, first say what the work was meant to achieve. DriftEvaluator makes that declared goal testable through Cases and Rubrics, then binds each trial to the exact subject revision being examined. Declared Evaluators compare retained observations with those criteria and return only findings the evidence supports. Their findings say where actual behavior and outcomes met or departed from the goal, and preserve what remains uncertain. ADR 34 owns this evaluation jurisdiction.

That office carries no authority to execute unsafe work, select or grant ana AnimatorModel Runtime, authorize spending, repair or mutate a subject, publish, train, or promote. Its findings become input to other owners, never their decision.

DriftEvaluator is Designed; no runnable evaluation path is delivered. State of Work records its maturity boundary.

Make the goal testable

A workflow may deliver the right artifact while exceeding its declared budget or omitting a required check. Its Case and Rubric must name those obligations before execution. DriftEvaluator then reports where behavior met, missed, or left the criteria unresolved, retaining the evidence for each finding. A later candidate can improve one dimension while regressing on another.

The same method applies to a first attempt: the declared goal supplies the reference. Historical comparison is optional, and several criteria need not collapse into one distance or score. Trials shows how to fix that reference before the result is known.

Execution is not verdict

A trial can be blocked by policy before it produces the evidence needed to judge its subject. Its execution status records that block; it cannot turn the absence of a trial into claim failure. Trials supplies the separate status and verdict vocabularies, including how to distinguish infrastructure, subject, and evaluator failures.

A Trial SuiteApp Bundle (TrialSuite@1) groups Cases and evaluation controls; a Composition Suite coordinates separately owned applications. These remain separate contracts.

A favorable verdict supplies no permission, routing authority, causal proof, or automatic promotion.

Choose the evidence passage

Start with Trials when constructing versioned Cases, controls, observations, Rubrics, calibration, repetition, and immutable Outcomes.

When healthy Outcomes must become scoped, expiring evidence for DispatcherTraffic Controller, Toll, or SoulforgeModel Trainer to consider under their own authority, continue to Capability claims.

Use Returning findings to trace a downstream failure across an exact CompositionApp SuiteApp Bundle and return inert evidence to the smallest supported boundary, without initiating reverse execution.

DriftEvaluator locates the departure from the goal only as far as evidence reaches. The receiving owner decides whether and how to correct it.

Evaluation can inform Creation before promotion and after use. A trial of the software candidate asks whether it meets its goal; a trial of the workflow asks whether the method produces better work. Workflow improvement separates those subjects and follows the evidence into an owner decision. Ouroboros places both passages beside memory and training.