Glossary

The vocabulary used throughout these docs, from what a client sees to what a method implements.

Scenario #

One workload: its records, training state, and release chain, isolated from every other scenario. The first request carrying a new x-reef-scenario creates it and permanently binds its recipe. The request may also bind a starting release.

Agent #

The complete system that acts: model + harness. The model supplies the learned behavior; the harness supplies the control loop and context around it. Reef improves an agent by updating either model weights or the harness tree.

Agent record #

One stored exchange, either an inference call or a report, with its id, scenario, request type, payload, and, for inference, the release that served it. The record store lives under agent_record_dir.

Receipt #

The id of a stored inference record, returned in x-reef-agent-record-id or, for a stream, as reef.agent_record_id in the terminal SSE metadata. It names the exchange and the release that produced it. Reports quote receipts in references.

Report #

A POST /reef/report request carrying feedback about one or more exchanges. Reef consumes each report at most once, including when it arrives late or is retried.

Feedback #

What you send back about a served response. A report carries it in two fields: score, the numeric channel, and feedback, a string or structured object such as a rubric breakdown. Reef's core stores both without interpreting them; the recipe decides what they mean.

Recipe #

The method a deployment runs. It binds a processor, a training objective, a runtime, and a surface. The core recipe records without producing updates.

Recipe reference #

What recipe.implementation selects. recipe is the core record-only implementation; a dotted package.module:ClassName selects an installed method class; any other bare name resolves only to a YAML preset under REEF_RECIPE_CONFIG_DIR. Reef has no global recipe-implementation registry.

Artifact #

The selected content: a model checkpoint, live weights, or a harness tree. content_id identifies those contents independently of when or why Reef published them.

Release #

One accepted publication decision in a scenario. release_id identifies the link in the scenario history and parent_release_id identifies its parent. Rollback therefore creates a new release that can point to the same content_id as an older release; history is appended, never rewound.

Content ID #

The identity of the bytes or logical model state selected by a release. Equal content_id values mean equal selected content even when the release_id values differ, such as after rollback or republication.

Release chain #

A scenario's history of accepted updates, each release carrying its parent. A durable release can be restored when its surface and runtime support rollback.

Component #

One named part of a release's content, such as weights, harness, or skills. A recipe's surface declares the components its releases bind and how each is admitted, loaded, injected, and read. A one-component release is flat; a release with several keeps one directory per component and a manifest naming each component's content_id.

Candidate selection #

The decision whether a produced update should replace the current served release. A CandidateEvaluator measures the candidate, a CandidateSelector makes the select-or-reject decision, and a CandidateEvaluationPlugin implements both. The docs also call this the gate. Every recipe carries the plugin as candidate_evaluation. It is independent of checkpoint cadence and retention.

Surface #

How a published artifact reaches whoever uses it: runtime loading, inference hooks, and an optional client-pulled file tree, composed as fields on one Surface value.

Training pipeline #

The method's data side. Three stages turn records into a typed training batch: the record-to-record stage decides which records train, the record-to-task stage turns each record set into tasks, and the task-to-batch stage shapes the tasks into a batch. Reported feedback uses valid reports with existing inference references; computed feedback derives its signal from traffic. Reef's TrainingPipeline drives the stages and owns batching, consumption, and retention.

Training objective #

The method-owned TrainingObjective declares its backend loss family and prepares a complete reserved batch. Its prepare method returns advantages, metrics, and proposed algorithm state in a StepSignal before optimizer or worker partitioning. It imports no training stack. A recipe selects it by registered name or dotted class/instance path and binds the StepScheduling the runtime cuts each batch with; the objective only rejects a schedule its loss cannot train.

Loss family #

The backend implementation selected by TrainingObjective.loss_family and exposed through WeightTrainingSpec.loss_family. It computes model-dependent terms and the tensor loss. Shipped method families include sao, tttd, and openclawrl.

Harness #

The part of an agent around the model. It owns the control loop, prompts, conversations, tools, environment integration, and grading, and calls the model through an inference runtime.

Harness tree #

Reef's versioned representation of the mutable files in a harness: config, rules, prompt templates, skills, extension code, and, for the native adapter, the loop's own tools, hook listeners, and control flow graph. Reef serves it over GET /reef/harness. An adapter combines the rendered tree with a harness executable; that harness and its configured model form the running agent.

Render #

Turning a harness tree into the files one agent uses: harness tree entries in, paths and their text out. This process does not touch disk or start an agent.

Reef's shared engine and the adapter own it together. The engine goes first, merging and writing by rules that are the same for every agent and rejecting a tree that breaks them. The adapter gets the last word: its paths decide where each file goes, and its quirks module may rewrite, drop, or refuse what the engine produced before an agent ever reads it.

Skill #

A SKILL.md instruction file the model reads, stored under skills/ in a harness tree. Reef can inject one into each request or let a client pull it with the rest of the tree. Updating one needs no GPU.

Runtime #

InferenceRuntime executes inference requests and owns admission; TrainingRuntime prepares training jobs and exports checkpoints. They are independent components. The existing RuntimeCandidateBackend coordinates candidate activation, publication and recovery for weight recipes. Concrete backend integrations implement the component operations.

SAO #

Single-Rollout Asynchronous Optimization (arXiv:2607.07508), the cookbook sao recipe. One graded rollout drives one training step, with no comparison group and no slowest-sample barrier.

TTT-Discover #

Test-time-training discover (arXiv:2601.16175), the cookbook tttd recipe. A model specializes to one hard problem during the search; the search loop stays in the harness.

OpenClaw-RL #

The cookbook openclawrl recipe (arXiv:2603.10165). It trains an unmodified agent from live conversation traffic using a next-state binary reward, with no external grader or training report. A harness-provided x-reef-tag-session value is the supported way to correlate a conversation; it stays stable within that conversation and is not reused for another one in the same scenario. Transcript matching is the fallback when the tag is absent.

Runtime load ID #

An engine-scoped <incarnation>:<sequence> token naming the concrete weight load that produced a span of generated tokens. It is a serving fence, not a publication or content identity. Reloading the same content_id can produce a new runtime load ID, while rollback can create a new release_id for old content. A restart invalidates runtime load IDs but not the release chain.

Staleness #

The number of versions that a scenario served after the runtime load that produced a sample, bounded by max_staleness. Zero requires a sample to train against the weights that generated it. The record-to-record stage checks it before a batch forms.

Reported and computed feedback #

Two ways a signal reaches a processor. Reported feedback arrives explicitly, in a POST /reef/report. Computed feedback is reconstructed by the processor from recorded traffic alone, with no report on the wire. The choice picks the processor engine.

Method #

The learning algorithm itself. A recipe is the object that binds one method to a scenario; harness_evolve is one mechanism running many possible methods.

Adapter #

Two meanings. For harness evolution, the descriptor that combines a rendered tree with an executable to make a running harness (Harness adapters). For weight serving, a LoRA/PEFT adapter layered on a frozen base model.

Episode #

One headless run of the agent under test against one task, in a throwaway root holding nothing but the rendered tree. Harness evolution scores every task twice per step, once per side.

Mutation #

One proposed edit to a harness tree: create, update, or remove on a single root-level entry. A sequence applies as one composite proposal under one verdict.

Preset #

A recipe configuration file, <name>.yaml under REEF_RECIPE_CONFIG_DIR, named by recipe.implementation. It carries the recipe's own sections; it is not a deployment config and reef serve -c cannot read it.