Design

This page explains why reef is shaped the way it is. To operate it, see deploy and train.

A normal inference endpoint is stateless. It answers a prompt and forgets which artifact version produced the answer, which other exchanges belong to the same workload, and what a grader said about it an hour later. reef supplies that missing connection and nothing more. It sits in front of a provider-compatible endpoint, records which exact artifact version served every response, and accepts feedback that quotes those responses. When the scenario's recipe says so, it turns eligible records into the next version of the thing being served. Prompts, tools, environments, and graders stay where they are.

The core loop#

The inference runtime is always required. The training runtime exists only for recipes that change weights. Recipes that change text artifacts run their step in reef's own process, with no GPU involved. The wire details of the first six lines are in connect your harness, and what fills the opt block is in define a recipe.

The version is frozen before the model is called. RequestService reads the scenario's current artifact ref, builds the request against it, and stores the exchange with that exact ref. An update landing mid-request cannot change which version the receipt names.

A response is validated before it is recorded. The surface fences the request path from both ends. prepare_request transforms the outgoing payload, and what it returns is both what gets forwarded and what gets recorded, so training and replay see what was actually served. validate_response then checks the provider's answer against the frozen version. Raising there means nothing is recorded and the client sees the error, so a generation from the wrong weights never enters the store. For live weights the check is exact. reef asks the engine to echo its serving version and compares the two. If the engine reports a different version, an update overtook the request, so reef retries the whole request against the new head with exponential backoff and gives up with HTTP 503 at the retry deadline. If the engine reports no version at all, that is a backend contract violation and reef answers HTTP 409 immediately. Pass-through streaming is the exception. The engine cannot echo its version and stream at the same time, so WeightSurface.prepare_request leaves return_meta_info off when stream is true, and a plain SSE exchange is recorded unverified. The training backend avoids this by buffering. Its stream carries the complete response in record_response, and start_stream validates that against the frozen version before anything is recorded.

The record model#

Everything reef stores is an AgentRecord: an id, a scenario, a request type (inference or report), the payload, and, for inferences, the frozen artifact_ref. Records live in a SQLite store opened with WAL journalling and synchronous = FULL when a path is configured. The default in-memory database is for tests, so a deployment that omits the path keeps nothing.

The id is the receipt. It comes back on every inference response as x-reef-agent-record-id, and feedback quotes it. A report is a record whose references name the exchanges it judges. score is optional and must be numeric when present. feedback is a string or an object, and reef's core treats it as opaque: a rubric breakdown, judge output, or whatever the recipe reading it cares about.

Appends are content-checked, so the same id twice with different content raises a conflict instead of overwriting what was there. Consumption is once-only, because a separate table remembers the ids a batch consumed, so a grader retrying its POST, or a report arriving after its references were trained on, is absorbed rather than counted twice. Those two properties make the store safe to feed a learner from, and they are why a recipe never writes dedup code. Deletion happens only through compaction, only for rows the processor declared releasable, and the releasable set is recomputed from live state on every read.

Scenarios#

A scenario is one isolated lane of learning, holding a single workload's records, trainer, and version chain. Two scenarios can never share data or influence each other's updates. The first request creates the scenario and must name its recipe, and that request may also pin a starting artifact version. Those bindings are frozen for the scenario's lifetime and stored durably, so later requests omit the headers, including requests made after a restart. Naming a conflicting recipe later returns HTTP 409 instead of silently rebinding. What the recipe selects at construction is likewise immutable: a surface, a runtime, an inference backend, and an optional report schema.

Where the GPUs live forces one process-wide rule. A process runs at most one training scenario, and a single serial thread drains it, so prepare, remote execution, and commit never interleave. Scenarios that update nothing, or that update text artifacts through an in-process backend, are unlimited and commit inline on the accept path.

Surfaces#

A published version does nothing until the process serving it picks it up. A surface is how each artifact kind gets picked up. It never proposes, evaluates, or decides.

The surface is its own package because it is called from two directions. prepare_request and validate_response fence the request path, as above. On the commit and rollback path, validate shape-checks an artifact before staging, restore activates a rollback target, and reconcile_serving decides at startup whether the recovered head can still be served.

WeightSurface is the hard case. Weights live in engine memory between checkpoints, and weight versions are session-scoped. A recovered live head is therefore checked before it is used. If the engine reports a different serving version, or the head was never made durable, serving falls back to the last checkpoint and the next training step republishes a live head.

The version chain#

Every accepted update is a version with a parent. Versions come in two kinds, and what separates them is whether the bytes are durable. Identity is durable either way.

The diagram shows two live steps, and any number may occur between checkpoints. Cadence controls when live weights become durable. It does not control how often they change. A live version is a LiveWeightArtifactRef whose weight_version is an opaque <training-group-incarnation>:<update-sequence> token, with the incarnation keeping tokens unique across training-group restarts. Its record is durable while its bytes are engine-local, which is why the dashed restart edges point back at the checkpoint. The step counter, algorithm state, and record progress survive a restart. The weights in engine memory do not.

Durable versions are Git-backed, one ref per scenario, with LFS patterns for weight files and a reef-artifact.json manifest committed into every version. Heads move only by compare-and-swap. advance_current takes the head it expects, publish takes the parent it expects, and the push carries a lease, so a stale publication conflicts loudly instead of losing a version. A staged process-local version (local: prefix) has identity but no durable bytes, and recovery and rollback refuse it as a restore source. Rollback never rewrites history. Restoring an earlier version activates it through the surface and republishes its bytes as a new commit, with the step counter still increasing.

Durability#

The whole guarantee rests on a per-scenario append-only JSONL commit log, where the fsynced append is the commit point. Each committed step writes one record, holding the step number, the artifact ref, the checkpoint flag, the algorithm state, the record high-water mark and the ids its compaction deleted, and the step's metrics. Every other store is derived from that log.

The ordering around that append is fixed per step kind, so a crash in any gap is recoverable by replay:

A checkpoint writes to the repository first, so a head ahead of the log is adopted and the healing record appended. A crash between append and compaction re-applies the recorded deletions. A staged ref is process-local and never a recovery source, so a local step may move the head first, and losing both together means the step never committed. A torn last line is read as a crash mid-append. Corruption elsewhere, or a gap in the step sequence, is reported as an error rather than guessed at.

Runtime work happens outside any transaction, so the runtime's training step must succeed before the batch is acknowledged. Publication comes after the trainer commit, and a publish that fails discards the staged artifact and reloads the scenario from durable state, so the uncompacted records are replayed. One logical reef writer per scenario is required, and external training operations must be idempotent or reconcilable across crashes. All of this rests on configured storage. The service defaults its record store, commit logs, and Git repository under .reef/, and a deployment that points them somewhere ephemeral gives up every guarantee in the table above.

The boundary#

reef versions and updates served artifacts, meaning weights, skills, harness trees, and playbooks. Run state, the bookkeeping a harness produces while executing a fixed algorithm, stays outside.

Your harness owns prompts, conversations, tools, environments, and grading. reef owns scenarios, records, serving-version tracking, trainer coordination, and the version chain. The runtime owns inference, plus GPU training when a recipe needs it.

What decides the split is the direction of data flow, and how agent-like the code looks has no bearing on it. Code whose input is a request or the environment and whose output is the next request is a harness, however sophisticated. That covers orchestration, grading, and search loops, and it is why TTT-Discover's PUCT search stays outside. Code whose input is the record store and whose output is a publish on the version chain is a recipe backend, even when it calls an LLM to do its thinking. Such code never answers a user request and never touches the environment. One technique can have both ends, as TTTD does with its search outside and its grouped training step inside, and the seam between them is the same in every case: receipts go out and reports come back.

Gating is a separate question from the boundary. No update is guaranteed to be an improvement. A weight step can regress, and the openclawrl and sao recipes publish every batch with no gate at all. What protects them is the chain: every publish is a commit, every receipt ties an outcome to the exact version that produced it, and any earlier checkpointed version can be restored where the runtime supports it. Live, uncheckpointed versions are refused as not restorable. Weight rollback additionally needs a runtime that implements restore_checkpoint, and the bundled Ray/slime runtime does not, so today rollback is a harness-artifact operation. Whether a recipe gates before publishing is a per-recipe risk policy. Any served artifact can have one, and none is obliged to. The full argument, including the cases it was written to settle, is in docs/rfcs/evolution-boundary.md. Terms used here are defined once in the glossary, and what ships is in the recipe catalog.