# HarnessKit > Eval harness and ship-readiness scoring for AI agents. Define an eval suite (JSON), run it against a recorded agent trace (JSON), get a weighted score and a SHIP / REVIEW / BLOCK grade. ## Status v0 preview (2026-09-25). The eval engine runs in the browser against recorded traces. The hosted runner (executing suites against live agents on HarnessKit infra), hosted CI gating, SDKs, accounts and team dashboards are not built yet and are labeled as such on the site. ## What it does - 19 check types: content (contains / contains_any / contains_all / regex variants), structured output (json_valid, json_has_keys), behavior (tool called / not called / sequence / argument match / no tool errors), budgets (steps, tool calls, tokens, latency, cost). - Weighted scoring with critical checks that block a run regardless of score. - Grades: SHIP (>=90%), REVIEW (70-89%), BLOCK (<70% or any critical failure). - Local CLI (`node cli.js suite.json trace.json [--fail-on block|review] [--junit FILE]`) runs the same engine in your own CI job; exit 0 = gate passed, 1 = failed, 2 = usage error. - Imports Anthropic Messages transcripts, OpenAI Chat Completions messages and OpenTelemetry GenAI spans (converted client-side, or by the CLI). - Starter suite templates: support agent, RAG answerer, coding agent, structured extractor (each with a sample trace). - Suggest checks: drafts a starting suite from a loaded trace (tools, order, JSON shape, budgets at 1.5x observed). - Batch runs: an array or JSONL of traces gives per-trace grades, the mean score, the worst trace and per-check failure counts. The batch grade is the worst trace grade. - Baseline diff: compare a run with a saved report to see regressed, fixed and became-unknown checks and the score delta. The CLI has `--baseline FILE --fail-on-regression`. - Missing trace metrics make a budget check UNKNOWN (excluded from the score, caps the grade at REVIEW); malformed checks are ERRORs and count as failures. - Complementary to ConeReplay (trace replay for model-swap regression prediction): replay with ConeReplay, score with HarnessKit. ## Links - Home: https://harnesskit.app/ - Docs (trace schema, every check type, scoring, CLI): https://harnesskit.app/docs.html - About: https://harnesskit.app/about - Contact: https://harnesskit.app/contact (info@persooninc.com) ## Company HarnessKit is built by Persoon.ai Inc., a Wyoming corporation (https://persooninc.com/).