Choose a recipe for agent learning

Pick a recipe by the task type of your workload and by what it evolves, model weights or the agent harness. Weight recipes need GPUs and the training stack in Train model weights from agent feedback, while harness recipes need only a model endpoint.

Reefine ships with reef-infra, and the other implementations live in the repository's recipes/ cookbook and do not ship in the Reef wheel. recipes/basic/ is the record-only starting stack and stays outside the catalog, and beta recipes join it once they publish learning results. The root README and recipes/README.md show the same catalog.

Scientific discovery #

One hard problem with a measurable objective, where the recipe makes repeated attempts and trains on those attempts at test time.

Continual learning on a task stream #

A stream of independent tasks that a verifier scores one by one, so the recipe learns from each score before the next task arrives.

Learning from usage #

Real interaction where no one reports a score or the feedback arrives late, so the recipe reads the signal out of the traffic it already serves.

How a recipe is selected #

A deployment serves exactly one recipe, named by recipe.implementation in its config. Every scenario it creates uses that recipe. Requests never name a recipe, and scenario snapshots do not store one. The scenario header is the only routing a caller provides. The artifact repository is therefore deployment-owned: do not point deployments configured with different recipes at the same repository.

recipe:
  implementation: recipes.sao.recipe:SAORecipe
  config:
    batch-size: 1

recipe.implementation accepts the core value recipe, a dotted class, or a preset. Reefine ships in Reef, and other learning methods are imported only when selected. The recipes/ tree in this repository is a cookbook; installed method packages work the same way. Configuration describes each spelling.

Every recipe has a checkpoint strategy, defaulting to EveryNVersions(1). checkpoint_every_n_versions is the shorter spelling in deployment YAML.

Run the recipe you chose #

Start with the inference and feedback quickstart if you have not sent traffic through Reef yet. For a harness recipe, follow Evolve agent prompts, rules, and skills. For a weight recipe, follow Train model weights from agent feedback. Each recipe page above provides its own configuration and example.

Built-in harness refinement #

Reefine turns user instructions into harness updates and ships with reef-infra. Start it with reef serve --recipe reefine and a configured model endpoint.

Beta recipes #

CORAL TTT and its coral_demo example are beta until complete, reproducible learning results are published. Both live under recipes/beta/coral/. Integration and smoke tests validate the wiring but do not establish learning performance.

SPADE (self play in adaptive synthetic executable environments, arXiv:2608.19197) is beta under recipes/beta/spade/: one policy plays an Environment Designer that writes executable environments and a Reasoning Agent that learns in them. Reef knows one task format, Harbor, and the Designer writes it directly: an instruction, a container, a verifier and a reference solution, a task any Harbor agent can play. SPADE is a task generator. The deployment selects it as the generator of the generated-task stages in its pipeline section, and the built-in generated-task recipe plays the tasks and trains on them (see generated tasks). Writing, checking and playing tasks is Reef's (reef.record2task.generator, the generator service reef serve starts beside the HTTP service). The package holds the method: the Designer's adversarial experience section, the hint arm each task asks for, and the hint based regret reported against the Designer's receipt. The Designer's own training follows.

AutoEnvScaling (preprint forthcoming) is beta under recipes/beta/autoenvscaling/. It is a task generator: a coding agent rebuilds recorded coding-agent sessions as Harbor tasks. The agent, pi in an E2B sandbox, writes one task in its workspace from one session and the repository the session changed. It runs the task's oracle and nop check itself through the generator service, and revises until the check passes. A model call reviews each task's tests and refuses a task whose tests check what its instruction does not state. A pass-rate band marks the tasks the Reasoning Agent can reach. The deployment selects it as the generator of the generated-task stages in its pipeline section, and the built-in generated-task recipe plays the tasks and trains on them (see generated tasks). With author: prompt, the generator service's one-reply prompt Designer rebuilds the same sessions as the baseline.