Events

If your experiment is written in Python, use the reps_events library:

from reps_events import Result, Status, emit

emit(Result(name="score", value=1))
emit(Status(detail="Starting evaluation"))

For experiments written in other languages, invoke the reps-emit CLI directly:

reps-emit result --name score --value 1
reps-emit status --detail "Starting evaluation"

Both APIs validate events, deliver them to the runner, and report delivery errors. To execute an experiment, use its named app (for example, nix run .#inspect-hello -- ...). The generated experiment launcher starts the runner and configures event delivery. Do not invoke reps-runner or the underlying experiment program directly to start a run, and do not implement a socket client in experiment code.

The experiment program’s stdout and stderr are captured as text, including printed JSON. The runner supplies the transport configuration automatically. See emission tools for CLI flags and Python emission for model construction.

Each run stores event envelopes in events.jsonl. The runner owns the envelope and lifecycle events. The following sections describe the saved format and event vocabulary.

Transport

{"v":1,"ts":"2026-01-01T12:00:00.000000Z","run":"RUN_ID","experiment":"example","schema":0,"seq":0,"event":{"type":"result","name":"score","value":1}}

Payloads use the closed set of types below. Unknown types, missing types, undeclared fields and malformed payloads are invalid. The runner rejects invalid socket submissions. All captured stdout/stderr remains text inside standard events, so saved records conform to the typed Python reader. Put experiment-specific JSON inside CustomEvent.data.

Lifecycle events

These types are generated by the runner. Experiments must not use the run. prefix for their own types.

run.start records the identity, inputs and run seed described in run files. params is the bound input passed to the process. Experiment identity is on the envelope, not duplicated in the start payload. There is no dirty field.

fetch_ref is a pinned clean repository revision; its Python value is None and its wire field is omitted when no such revision is known. Launchers reject fetch references containing URL userinfo. tree_hash is the packaging tree’s NAR hash, emitted whenever the launcher knows it, for clean or dirty trees. It is separate from experiment source identity.

runtime records platform, runner_python_version, cpu_model, cpu_count (logical cores available to the process), and endpoints. Hardware fields are absent on older records. Executable fields experiment_bin and runner_bin are recorded only for /nix/store/ paths; other paths are omitted. Endpoint keys are the configured provider sets used by this run; values are only scheme://host[:port], with no userinfo, path, query, or fragment. The child still receives the full configured URL. This is a runtime snapshot, not a process-environment dump.

result_definitions snapshots the manifest’s ordered list of named result declarations, including labels, descriptions, details and units when supplied. The same definitions are copied into run.json.definitions.results.

run.end contains only process facts: terminal state (completed, failed, or interrupted), duration_s, and exit_code. The exit code is the child’s return code; negative values identify signal termination. An incomplete stream may have no run.end. Consumers derive summaries from the last result per declared name and sum usage over llm.call records; the runner caches both in run.json.derived, while run.end stays free of derived values.

Producer toolchain

Python producers emit producer.python once per run, before their other events. reps_events.emit_producer() records the running interpreter; experiment_main and the Inspect entry point call it after validating params. Other Python entry points call it themselves. This is producer evidence, separate from the runner’s launch snapshot in run.start.runtime.

reps-runner verify checks that this event is unique and precedes other producer submissions. Asynchronously captured stdout/stderr may precede it. For Nix runs, it checks the executable’s store prefix against the launcher’s Python venv, following static wrapper references without executing them. It reports a skipped containment check when the venv cannot be recovered. Older runs without the event produce a warning and remain verifiable. Future language-specific producer.<language> tags belong to that language’s events library.

Status and logs

status describes experiment progress and does not change the lifecycle state. stdout and stderr are generated by process capture and have no inferred severity.

Results

A result has required name and scalar value (finite number, string or boolean). The name identifies a result declared in the manifest. Units and explanations come from the declaration, not from the event. An undeclared name warns and stays out of the summary. A repeated declared name warns and its last value wins. Missing results stay absent. Every event remains in the transcript. Series and per-item outcomes use experiment-specific custom events.

Experiment-specific observations

Model calls use the shared llm.call event across experiments. Other observations, including communication, use CustomEvent where we have not established a shared meaning. We are deliberately deferring further abstractions: similar-looking concepts can mean different things in different experiments.

Each experiment defines its own custom kind strings and JSON data objects. GovSim preserves its upstream rows as govsim.harvest, govsim.utterance, govsim.summary, and govsim.resource_limit; unrecognized actions remain govsim.record. A govsim.upstream_log marker identifies each ingested file by relative path, byte count, SHA256, and record count before its rows. Persona nodes.json rows become govsim.memory with a persona ID and the untouched node. Embeddings are omitted because node descriptions and the embedder named in govsim.config reproduce them. Those kinds do not define a communication convention for other experiments. The default reader uses generic CustomEvent objects. Pass an experiment Payload union to read_events to restore its typed custom data.

Event classes may declare render: ClassVar[RenderHint]. A hint has a Lucide icon name and optional dotted payload paths actor and actor_label. title and body accept a bare path or literal text with {dotted.path} substitutions; missing values become empty strings. Templates have no expressions, conditionals, functions, or defaults. format is text, markdown, json, or html-text; the last strips HTML tags and displays plain text, never foreign HTML. Optional fields are ordered dotted paths shown as an expanded definition list, omitting absent paths. A truthy badge path displays its last segment as a badge. The raw disclosure retains all original data. An optional actor_registry declares a path to a map of actor IDs to entries and a label field within each entry. export_schema(Payload) embeds hints under x-reps-render; they never enter event records. Literal-dependent hints use JSON Schema if/then clauses, for example log severity and the stdout/stderr channel.

Readers can resolve hints from the exported schemas. actor identifies an actor; actor_label supplies its display name, and registry events can provide labels for actors referenced elsewhere in the run. The original event payloads remain available regardless of presentation hints.

Round or step boundaries can be custom events in source order. Mark post-run replay segments explicitly to distinguish them from earlier live calls.

Model calls

An llm.call records one observed model API operation and its available outcome. SDK-internal retries may be invisible to the adapter; this is not necessarily one HTTP attempt. It is not an agent turn or a message sent between agents. Request messages record the input supplied by the caller, not proof of the provider’s complete internal context. Tool requests inside a response do not establish that those tools were executed.

llm.call retains boundary data from Inspect’s ModelEvent, using locally vendored Pydantic models from Inspect AI 0.3.263. The source version, archive hash, local changes and MIT license are recorded alongside reps_events.inspect_chat. The event library depends only on Pydantic. There is no tracing runtime.

SDK retries and empty-response retries are counted in the same retries field, so one record can exceed the per-layer cap: up to 8 attempts in the empty-response loop, each allowing 9 HTTP attempts (the initial request plus 8 SDK retries), for at most 72 HTTP attempts. Exhausted HTTP 429/5xx retries stop at the SDK budget; the client does not start a second cycle after those nine rejections.

Messages have system, user, assistant or tool roles. Content is text or typed blocks for text, reasoning, images, audio, video, documents, server-side tool use, or provider data. Tool calls, tool errors and citations retain their Inspect data fields relevant to the boundary. Tool error types are open strings; boundary producers use what the API reports (Anthropic’s error flag becomes "error"), and harness adapters can retain native error types. Parsing errors remain on ToolCall.parse_error; message source and tool viewer hints stay in raw harness records. Message IDs are preserved when supplied; the vendored models do not invent IDs. Provider-specific fields that do not fit these models remain in call.

output.usage keeps input/output/total tokens, cache-read/write tokens, reasoning tokens and cost. Inspect’s input_tokens excludes cached tokens; consumers add cache-read/write tokens when computing total input usage. Missing usage remains unknown. The shared usage model defaults absent token counts to zero, so the OpenAI-compatible converter leaves output.usage absent when required counts are missing, retaining all reported partial usage in call.response. output.model records the returned model; the SDK’s requested model remains in call.request when available.

The served name must equal the requested name ignoring case, except that an unqualified or -latest request also accepts the same name plus one date suffix (-YYYY-MM-DD, -YYYYMMDD, @YYYYMMDD, -YYYY-MM; -YYMM for mistral/ only).

A response with an empty model ID and a nonempty set of entirely content-filtered choices is exempt from the identity check. Verification warns once per run that model identity could not be checked on content-filtered calls. The client keeps its first-response identity check pending for the next response with a model name. Other empty or non-matching served IDs are mismatches. Version 0 records with empty choices and no error are read as failures with Empty choices for model '<model>' (version 0 record, not retried). Verification fails these runs, including when the empty response supplied a model ID.

Read generation settings, including temperature and token limits, from call.request. They are not projected onto separate fields. Inspect’s GenerateConfig is retained only in the raw inspect.event record. The OpenAI-compatible client converts SDK messages, tools, choices and usage at the capture boundary and keeps the full SDK request/response in call. The first well-formed <think>…</think> block in assistant text is reasoning (Inspect’s rule), provider reasoning_content is reasoning, and everything else is text, untrimmed. GovSim trims at its Pathfinder backend for parity with local generation and anchored parsing. call.response retains the SDK response and its unmodified content strings, including HTTP error bodies. An error body containing valid completion choices uses the same output conversion as a 200 response and has no error; new records leave call.error unset (None). HTTP errors without usable output retain the body with error set and call.error unset. call.error becomes true only when the request failed without a captured body, such as a connection failure or timeout. Non-object HTTP bodies are retained under call.response.body. The response hook only counts HTTP 429/5xx responses. HTTP errors without valid choices retain the body and SDK message and are raised immediately. Only 2xx responses with empty choices enter the empty-response retry loop; exhaustion records Empty choices with the model, attempt count and last ID. The SDK’s own retry policy is unchanged.

parse_event validates a bare event payload. parse_record decodes a saved JSON envelope, applies reps_events.migrate_record(record) to the dict, then validates it with Envelope; read_events calls parse_record for each line in a file or run directory. Both readers accept an optional payload union or adapter, validated after migration. Migrations interpret version 0 error-body reprs and silent empty responses under the version 1 rule and set v to VOCABULARY_VERSION. They read v from the record, rejecting missing, invalid or newer versions without rewriting saved files. An absent call remains absent. Recovering a version 0 error body clears its stale call.error; converting its choices to usable output sets that legacy flag to false. The silent-empty-response migration leaves call.error as written. For prefixed version 0 errors, a body that cannot be parsed as a Python literal and JSON-compatible data, or whose choices cannot be converted, raises MigrationError; read_events wraps it in EventReadError with the line number. Errors without the SDK prefix remain unchanged. Envelope accepts only the current v and defaults to it when constructed. derive_card projects the records supplied by the reader without applying migrations itself.

Verification uses migrated records for structure, order, identity, failed-call counts, model checks and the rebuilt card, failing on any remaining llm.call.error with the count and first message. Its experiment-union subprocess validates raw lines using the build’s own reps_events, including the v0 model in retained old builds. An older card is accepted only if it exactly matches the original projection built from the JSON lines, or the migrated projection. Verification does not compare record versions with the manifest: the runner checks vocabulary compatibility before launch and stamps the manifest’s version when writing. Historical inline reasoning strings remain unchanged.

File deduplication references (input_refs), tracebacks, streaming state (pending, working_start), and Inspect’s event identity/timestamps are omitted from the payload. The existing envelope supplies record identity and capture time; it has no span_id. Completion-time emission still cannot record a call if the process dies while waiting for the provider.

Custom events and deferred concepts

A custom event has a namespaced kind and JSON object data. GovSim schema 0 specializes these with strict models and a union discriminated by type, then kind:

{"type":"custom","kind":"govsim.state","data":{"round":1,"resource":42,"collected":{},"limit":5,"final":false}}

Read GovSim with read_events(run_dir, payload=govsim_adapter.models.Payload). Generic readers still use CustomEvent. Inspect and Concordia use their own prefixed kinds, including inspect.sample, inspect.artifact, and concordia.provenance. Every Inspect transcript event is emitted as inspect.event, with its native JSON-mode dump (omitting absent optional fields) as data, in transcript order. Each model event is followed immediately by its derived llm.call, including cached events. Those calls use the producer notes inspect.sample_id, inspect.epoch, and, when applicable, inspect.cache in metadata; raw config, roles, retries and cache bookkeeping stay available in the native record. Shared instance attribution is deferred.

Shared instance, artifact, agent.event, tool.call, span, and limit models are deferred. File pointers belong in custom data; producers write the file before emitting its pointer. reps_experiment.deposit_artifact requires the custom kind explicitly and writes text files under the run’s artifacts directory.

Emission tools

reps-emit validates a producer payload, sends it to REPS_EVENT_SOCKET, and waits for acknowledgement. The reps-events package installs it without depending on the runner. It is also available in the runner’s environment through that dependency. The following checkout command only inspects a schema; it does not launch an experiment:

uv run --project lib/reps-events reps-emit schema llm.call

In the experiment program, invoke reps-emit result --name score --value 1 to record a value.

reps-emit schema prints schemas for all standard payload types, while schema TYPE selects one. schema --union exports the discriminated payload union; schema --envelope exports the complete saved record. The helper offers these emission subcommands:

llm-call emits the llm.call wire type. JSON-valued flags take JSON text; --value accepts JSON or a bare string. llm-call also accepts a payload body object on stdin, with flags overriding fields from stdin. Invalid standard payloads return status 2 and print an error to stderr.

Python producers construct Pydantic event models and call emit(event); see process protocol. Public producer classes and typed CustomEvent subclasses are accepted by emit. Unknown constructor fields raise; explicit metadata objects preserve provider and experiment fields. Use CustomEvent for generic experiment data.