Candidate Handoff

In plain words: Evaluating and promoting trained models

Designed 2 min read 6 glossary terms 8 connected pages
The short version written for the Snayu team

How a frozen trained candidate is independently evaluated, how an external decision may promote it with explicit limits, and how it is watched, rolled back or invalidated later.

Why read it: Read this before putting any fine-tuned model or adapter into use, or building model promotion and rollback.
  1. 1Evaluation needs sealed holdouts, baseline comparison, regression suites and serving-cost measures; self-grading does not count.
  2. 2Passing evaluation means eligible only; a Promotion Decision names allowed uses, rollout, stop conditions and rollback.
  3. 3Files appearing in a folder create no capability; only the serving owner registers the promoted digest.

Designed Neither independent evaluation nor promotion runs yet.

Words on this page, decoded click one for its card

The original text Underlined words open their plain-English card.

The fire has shaped it; other hands will weigh it.

A frozen Candidate Bundle crosses from a Training Run into independent judgment. DriftEvaluator may establish eligibility; only an externally owned decision may promote the exact digest, bind its serving envelope, or refuse it.

Candidate handoff is Designed; neither independent evaluation nor promotion runs. State of Work owns delivery, and ADR 33 owns the handoff law.

Independent trial

Entry accepts one exact Candidate Bundle revision with its trainer receipt. DriftEvaluator names that revision as the subject and operates separately from the trainer. Evaluation cannot alter candidate bytes, corpus, Recipe, or lineage.

An expected-change contract recorded before training defines sought lift and tolerated regression. Any post-training change to that contract remains visible rather than silently reshaping the trial.

DriftEvaluator records compact promotion evidence under ADR 34:

  • sealed target-capability holdout results;
  • matched baseline Outcomes for the base model or currently promoted revision;
  • named regression Trial SuitesApp Bundles plus adversarial and authority-boundary Cases; and
  • quality, latency, memory, and cost in a compatible serving Environment, with uncertainty, errors, exclusions, and Evaluator calibration.

Trainer loss, development improvement, trainer-authored samples, and candidate self-grading remain diagnostics. Success on one visible Trial SuiteApp Bundle cannot establish general capability or safety outside its measured Environment.

Eligibility is not promotion

Passing DriftEvaluator establishes eligibility only. The owning policy and required MagusYou, the Operator/HitL authority choose whether to promote and which capability envelope may be exposed.

The external Promotion Decision states exactly what may enter use. It binds the Candidate Bundle to its independent Outcomes, names eligible capabilities and denied uses, and fixes compatible engines and the base relationship. It also defines how that use will be watched and stopped: rollout population, observation window, stop conditions, fallback, rollback, and retention, quarantine, or retirement terms.

The serving owner registers and routes the exact promoted digest. Files appearing in a directory create no capability, and DispatcherTraffic Controller cannot infer eligibility from their presence. SoulforgeModel Trainer neither promotes nor serves its output. Promoted revisions remain immutable; supersession requires a new Candidate Bundle and Promotion Decision.

Observe, roll back, invalidate

Versioned serving observation belongs to the declared evidence window. A stop condition or regression routes new work to an earlier promoted digest and quarantines the suspect revision. Rollback cannot erase completed effects or remove learned examples from produced weights.

If privacy, authority, license, contamination, or holdout failure is discovered later, follow its consequences through the retained lineage: Dataset Manifest, Training RunJob Record, Candidate Bundle, Outcomes and evaluation claims, and Promotion Decision. Policy determines which reuse or serving must stop, which controlled artifacts must be quarantined or deleted, and which evidence must be invalidated. A replacement requires clean admission and rebuild. Deleting the source record alone does not untrain weights derived from it.

Custody ends in one durable state: the exact digest remains promoted within its bounds, is superseded by a new immutable decision, or is quarantined and blocked with lineage intact.