agent-harness GitHub
Menu

Overview

agent-harness Agent Harness Blueprint Blueprint plugin product brief The Harness Book Deny-floor known limitations Licensing scope and decision Replay tool product brief Agent Harness roadmap Harness Specs

Docs

Advisory agent-operations measurements Mapped authority-document drift probe Mapped authority drift: pinned local trial Benchmarks and measured evidence Corpus path boundary Advisory entropy measurements Learning evaluator evidence boundaries Outcome stream validation System comparisons and canary evidence Advisory logical-job accounting Replay lab Python check inventory Replay CLI input identity and timeout limits Replay input decoding failures Replay report output boundary Replay process policy-root aliases Replay supervisor marker scope Review evidence and acceptance oracles Offline reviewer-packet diagnostics System state Process-fixture startup is not containment evidence

Agentic

Architecture additives — perfect assistant → estate owners Perfect autonomous agentic assistant — estate synthesis

Evals

Harness extension notes Evaluation merge-boundary contract: v0 Merge-gate model Outcome ledger Review under flood Agentic development evaluation taxonomy: v0

Evals / Examples

Worked evidence packet: a private product PR

Extraction

Replay v0 extraction inventory

Freeze

Legacy deny-floor v1 freeze record

Maintenance

Managed Codex agent publication Sync input and Doctor session roots

Observability

Coding-agent loop observability v0 Offline source-record identity inspector

Observability / Adapters

Claude Code to agent-loop v0: mapping note

Review evidence

Review/oracle wave: source ledger

UX evaluation

Frozen observation verifier Evidence-bound issue seeds Advisory product UX evaluation Offline scenario binder Scenario authoring contract v0 Taskdeck-first adapter sketch Workspace handoff acceptance profile Workspace receipt lab

Replay lab

See what a policy change actually changes. Explore 50 recorded public-fixture command cases through two versions of a toy guard. Follow a case, filter the differences, then adjust a sample regression gate.

The corpus has 40 synthetic cases and 10 fully re-authored historical/opaque shapes. This is a teaching example, separate from the current deny floor. It displays checked-in decisions; it does not run commands, contact a model, or measure a live deployment.

RECORDED EVIDENCE / PUBLIC FIXTURE

The interactive report is loading. The recorded summary and reproduction commands below work without JavaScript.

1. Follow a case

Guard v2 exempts help and dry runs, but accidentally drops its git clean rule. Start with the regression, then compare a benign improvement.

Choose a recorded case

2. Try a regression gate

Choose which transitions fail this sample gate. It always checks all 50 cases. These controls change only this browser demonstration.

Fail when a case is…

A transition is not a verdict about safety. Newly allowed help can be an improvement; newly allowed destructive cleanup can be a regression. The case rationale supplies that context.

3. Inspect the evidence

Filter recorded cases

What the recorded run shows

Transition Cases How to read it
Newly allowed 4 Three benign cases improve; one dangerous cleanup case regresses.
Newly denied 1 The candidate catches the corpus’s force-with-lease case.
Unchanged 45 An unchanged decision can still be wrong. Inspect the corpus label.
Newly indeterminate 0 Neither toy hook fails to produce a determinate decision here.

The default gate fails because it rejects any newly allowed case. These are the toy guards’ recorded decisions on charter-v0.1, not measurements of the current floor. The labels describe this teaching corpus; they are not universal shell-command classifications.

Sources: recorded report, toy guard walkthrough, baseline source, candidate source.

Reproduce it locally

From the repository root, Python 3.11 or newer can reproduce the comparison from recorded decisions:

python -m replay_v0.app replay \
  --baseline recorded:examples/toy-guard/report/baseline/decisions.jsonl \
  --candidate recorded:examples/toy-guard/report/candidate/decisions.jsonl \
  --corpus replay_v0/corpora/charter \
  --output reports/toy-guard-recorded

Expect exit 1 with newly-allowed: 4 in summary.json: the comparison ran and its regression gate failed. The toy guard walkthrough also shows how to invoke the two hooks afresh. Replay sends command text to the policies; it does not execute the recorded commands.

Put the tools together

Declare a policy, inspect its wiring, compare decisions and review the evidence

Use doctor and audit to inspect the declaration and wiring. Use replay to measure a proposed policy change on a pinned corpus. Review the changed cases before shipping. Runtime trust and live canaries remain separate evidence; see the floor limits and capability ledger.