What is REPS?
REPS packages replications of multi-agent safety experiments, runs them on your machine, and saves their inputs, results and execution records together. This repository provides the experiments, runner and Nix library.
What would you like to do?
| I want to… | Start here |
|---|---|
| Run an experiment without cloning the repository | Run your first local experiment: inspect inputs, run a keyless example and read its result. |
| Run from a terminal or use a copied command | Choose inputs and run: inspect the parameter schema, supply inputs and check the outcome. |
| Explore saved results | Read the files directly. |
| Repeat a run or try different inputs | Repeat and compare runs: keep the source and inputs, choose seeds and assess differences. |
| Add an experiment or change its code | Add or change an experiment: clone REPS, edit, test locally and submit a pull request. |
See the Roadmap for planned work on execution and recorded data.
How are experiments and results organized?
For example, choosing a model and dataset limit sets inputs for an evaluation. Running that configuration twice saves two separate records. Changing the model lets you compare another configuration. A run’s record can include messages, model requests and responses, scores, logs and files; the experiment determines what it emits.
An experiment defines a program and its inputs. A condition groups runs with the same experiment name, declared source content and complete set of input values. A run is one execution of a condition. Repeating a condition creates another run with its own ID and record.
For precise field and option definitions, use the references for commands, manifests, stored files and events.