Introduction

Reef is a continual learning infrastructure. It serves an inference endpoint in front of the model your agent already calls, records what it served, accepts feedback about it, and uses that feedback to publish a better version of the model weights or of the agent's harness. The model and harness together form the agent.

Nothing about the agent has to change except the base URL it sends requests to.

The harnessruns the control loop, prompts, skills, tools, and config

requests and feedback answers and a new harness tree

Reefserves requests, records results, trains weights, and evolves the harness

inference and new weights

The modelruns the weights: local engine or hosted API

Reef serves the model, loads new weights into the engine, and sends a new harness tree to the agent.

Why Reef #

Agents accumulate feedback that nothing consumes: a tests-passed signal, a thumbs-down, a rubric score. Turning that into a better agent normally means an offline pipeline: export logs, build a dataset, train, evaluate, redeploy.

Reef closes that loop in the serving lifecycle for continual learning.

Serveforward the request and keep the exchange

Recordstore the release that produced the response

Learnthe recipe uses records to create a candidate version

Publishan accepted candidate becomes the current version

the next request is served by the new version

Existing inference engines and RL frameworks cover parts of that loop, but not the whole thing:

Ability	Inference engine (vLLM, SGLang, …)	RL training framework (slime, veRL, AReaL, …)	Reef
Serves live traffic	✓	✗	✓
Trains weights	✗	✓	✓
Versions what it served	✗	✗	✓
Stays live through updates	✗	✗	✓
Evolves beyond weights (skills, harness)	✗	✗	✓

What Reef can evolve #

Model weights and the harness tree are the two artifacts Reef versions. Each updates one of the agent's two components. The recipe you configure picks one (composite version is a feature in the roadmap).

Model weights. Reef runs the training step and hot-swaps the result into the serving engine. See Evolve your model.

The harness tree is the versioned representation of the harness's mutable rules, prompts, skills, config, and extension code. Reef proposes an edit, runs the current and proposed versions on your tasks, and keeps the winner. See Evolve your harness.

What a call looks like #

A deployment names one recipe in its config. Every scenario it creates uses that recipe; recipe identity is not part of scenario state:

reef:
  recipe: recipe          # default recipe; stores records only;
  upstream_url: https://api.openai.com  # redirect to the OpenAI API

Quickstart starts one on a laptop. Reef's inference endpoint is OpenAI- and Anthropic-compatible, so a request to a served deployment is the one you would send to the provider, plus an x-reef-scenario header:

curl -sS -i http://127.0.0.1:8900/v1/chat/completions \
  -H "Authorization: Bearer $REEF_TOKEN" \
  -H "x-reef-scenario: hello-reef" \
  -H "Content-Type: application/json" \
  -d '{"model": "gpt-4o", "messages": [{"role": "user", "content": "hi"}]}'

The response carries x-reef-agent-record-id: the receipt naming the stored exchange.