Define an eval suite. Run it against a recorded agent trace. Get a ship-readiness score you can argue with. Every team shipping agents rebuilds this from scratch; HarnessKit is the kit so you don't.
This v0 runs entirely in your browser against recorded traces. Nothing leaves the page. A sample suite and trace are loaded - hit Run evals. Pick another starter template (support agent, RAG answerer, coding agent, structured extractor), or paste, load or drop your own.
Also accepts Anthropic Messages transcripts, OpenAI Chat Completions messages and OpenTelemetry GenAI spans (OTLP JSON). They are converted to a HarnessKit trace in the box.
A share link contains your full suite and trace, so anyone who has it can read them. The data sits after the #, which browsers never send to a server, so it is not logged or sent to analytics.
| Check | Status | Detail |
|---|
| Trace | Grade | Score | Failed checks |
|---|
final_answer_present - the agent actually answeredcontains / not_contains - string must (not) appear in an outputregex / not_regex - pattern guards, e.g. no SSN-shaped strings in any outputcontains_any / contains_all - one of, or every one of, a list of valuesjson_valid / json_has_keys - structured output parses (one ```json fence allowed) and has the required keys (dotted paths like order.id)tool_called / tool_not_called - required and forbidden toolstool_sequence - calls happened in the required ordertool_arg_equals - a call used the exact argument, e.g. the right order IDno_tool_errors - no tool step has an error field or status: "error"max_steps, max_tool_calls (optionally per tool), max_tokens, max_latency_ms, max_cost_usdweight; the score is weighted pass rate.critical fail the run outright (grade BLOCK) no matter the score.node cli.js suite.json trace.json, see the repo README).A trace is one JSON object: metadata (total_tokens, latency_ms, cost_usd), a steps array of {type: "llm"|"tool", name, output|result, args}, and final_output. The loaded sample is a complete example.
HarnessKit scores a recorded run against your expectations. It pairs with replay tools rather than replacing them: ConeReplay replays a trace through a causal cone to predict what a model swap breaks; HarnessKit then grades the replayed runs against your suite. Replay with ConeReplay, score with HarnessKit.
The category (LangSmith, Braintrust, Langfuse) is forming now and the comparison guides read like a menu. HarnessKit's bet: the harness itself should be a kit you install in an afternoon - open formats, no seat pricing, runner as the paid tier.