Bundled recipes

Choose a recipe by the signal your harness can provide and the artifact you want to update. When a scenario is created, x-reef-recipe must name the recipe loaded by the deployment. That binding does not change later.

Custom recipes use package.module:ClassName in the service config, but requests still use the recipe class's plain name. See define a recipe.

Every training recipe accepts checkpoint_every_n_versions (default 1), so the tables below omit it. Weight recipes accept their knobs as flat reef.<name> keys or as data.<name> in a named recipe config. Other recipe kinds read their knobs from data.<name>. The notes column names any available environment-variable fallback.

recipe#

The base class doubles as the bundled record-only kind: it ingests every request and report for audit, never becomes ready, and changes nothing. No training driver is involved. An inference runtime is the whole deployment, either a local SGLang server or a hosted provider through inference_proxy.

A skill: config section turns the same kind into skill delivery: skill.inject: true injects the scenario's versioned skill server-side into each request, false exposes it through GET /reef/harness for the client to pull. Nothing evolves the skill here. It is served at whatever version the chain currently holds.

Signal. The kind needs none. No report_schema is declared, so /reef/report ingress stays open, and anything posted is stored and never trained on.

Updates. Nothing changes. With a skill: section, the recipe only serves the skill version already selected for the scenario.

Knobs. There are none. What you configure is the runtime: section, the optional skill: section, and artifact.checkpoint_every_n_versions.

Pick it to prove an integration before any objective exists: bind a scenario, watch records land, and confirm that receipts and reports pair. It also covers pure skill delivery, and recording with a version chain but no training.

adapter_serve#

Inference-only serving for a PEFT adapter such as LoRA. An adapter trained outside reef gains versioning, pinning, and rollback without attaching a training backend. Reef routes every request to the adapter version currently selected for the scenario.

Signal. The kind needs none. No report_schema is declared, so /reef/report ingress stays open.

Updates. Nothing changes. Before publication, reef checks that the artifact contains a valid Hugging Face PEFT layout. It then loads and routes requests to the selected revision when the runtime supports adapter loading.

Knobs. What you configure is the runtime: section and the optional adapter: section. adapter.base_model is the guard worth setting: with it, publishing an adapter fit to a different base model fails validation instead of serving silently wrong weights. adapter.residency_window (default 4) limits how many published revisions stay loaded at once. A named recipe-config YAML in REEF_RECIPE_CONFIG_DIR looks like this:

kind: adapter_serve
runtime:
  type: inference_proxy
  base_url: http://127.0.0.1:30000
  timeout_s: 300
model:
  path: Qwen/Qwen3.6-27B
adapter:
  base_model: Qwen/Qwen3.6-27B
  residency_window: 4

Pick it to serve and version an adapter trained outside reef before any adapter RL training lands.

sao#

Single-Rollout Asynchronous Optimization (arXiv:2607.07508). One graded rollout is one training step. There is no comparison group and no slowest-sample barrier, so a rollout enters training as soon as its score arrives.

Signal. Each rollout needs one report with a finite score and references to the responses it grades. Any reward source works. The serving backend must also capture the original token ids and log probabilities; reef does not reconstruct them from text.

batch_size must equal the driver's --global-batch-size, because each rollout sample is its own data-parallel unit. An optimization: config section is rejected outright. The clipping bounds, critic cadence, and GAE parameters are owned by the backend and belong in training.slime_flags.

Pick it when the score comes back per task, sibling rollouts do not exist or need no comparison, and you want the policy moving without waiting for a batch to fill.

openclawrl#

Online RL from next-state signals (arXiv:2603.10165) for an unmodified agent. The whole recipe runs server-side. The processor rebuilds sessions from recorded traffic, judges each completed turn using the state that followed it, and turns accepted hindsight hints into a training signal.

Signal. The agent sends nothing on the wire. There is no report, no judgment, and no session header. report_schema is None, and reports are terminal on sight. The recipe reads only the traffic the agent already produces, so the agent points at reef directly and is otherwise untouched.

Only max_staleness (REEF_MAX_STALENESS, default 0) has an environment fallback, and the rest are config-only. batch_size must equal the driver's --global-batch-size, as in sao.

An empty prm_url is correlate-only. Sessions resolve and nothing trains, which is how you check trace matching against real traffic before standing up a PRM.

Pick it when the workload is a chat session and no grader exists. You cannot ask the deployment for a score, but the conversation itself says whether the last turn worked. It needs a second served model, the PRM, and in return it learns from traffic that ships no feedback at all.

tttd#

TTT-Discover (arXiv:2601.16175), where a model specializes to one hard problem during the search itself. The search loop stays in your harness, and the recipe handles the barrier and the grouped training step. Run one scenario per stated problem.

Signal. Each rollout needs a finite score and metadata identifying its step, group, and position in the rollout grid. The metadata also includes the configured group and rollout counts so reef can reject a report from the wrong grid.

rollouts_per_group must be at least two, because a group of one has no relative reward. The whole step is one barrier: no batch appears until every groups_per_step × rollouts_per_group slot is filled, a retried report at an occupied coordinate is terminal, and a step whose reports span artifact versions is discarded with every row released. Size the grid to what the harness will actually produce, or the step never completes.

Pick it when the harness runs a search that already emits a fixed grid of sibling rollouts per step and the comparison inside a group is the signal. The barrier costs you throughput, which is bounded by the slowest rollout in the step.

harness_evolve#

This is the only bundled recipe that evolves text instead of weights. Each training step, a propose callable reads the current composition tree and the batched traces and returns one mutation. The backend applies it, renders both the candidate and the current composition through an adapter descriptor, runs one headless episode per task against each, and lets evaluate score the results. The pairwise verdict settles it. A win publishes the candidate, and anything else reverts it. No GPU is involved.

Signal. Each recorded request needs one report carrying a finite score (ScoredRollout, the same contract sao declares) and exactly one reference. The recorded request itself becomes the sample, unmodified. Requests are recorded post-transform, so a trace shows exactly what the composition served.

Updates. A winning harness tree is published as a new version. Clients pull it through GET /reef/harness.

The score window's lower bound is unset, so the default keeps only traces at or below zero. Harness evolution learns from failures. This kind never reads the flat reef.* section, so all of its configuration lives in a named recipe-config YAML. batch_size and max_score go under data:, and the rest goes under evolution:, whose propose and evaluate may be dotted module:attribute strings so nothing has to be registered from code:

An empty tree renders an empty model config, so every gate dead-ties. The seed names the baseline nodes the first mutation is measured against.

Pick it when what should improve is the agent's prompts, rules, and configuration rather than its weights. You also have to afford running the task set twice per step, which is what the pairwise gate costs.

ace#

Agentic Context Engineering (arXiv:2510.04618) distills lessons from failed traces into incremental SKILL.md updates. The bundled implementation is not available yet; progress is tracked in #176.

Do not pick it yet. To evolve skills today, use harness_evolve with your own proposer.