ak eval

Use ak eval for the operation described below. The command help records its arguments, effects, and options.

Command contract

Run controlled skill and agent evaluations

Synopsis

Run trusted local suites in isolated workspaces and runtime homes. Graders check output independently; model calls may incur provider charges.

Examples

  ak eval run suite.json --json
  ak eval run holdout-suite.json

Options

  -h, --help             help for eval
      --json             Emit machine-readable JSON (implies --no-interactive)
      --no-interactive   Disable interactive prompts (CI-safe)
  -q, --quiet            Suppress non-error output on stderr
  -V, --verbose          Extra diagnostic output on stderr (loses to --quiet)
  -y, --yes              Assume yes for all prompts

SEE ALSO

• ak - AgentKit CLI
• ak eval decision - Compare an explicitly enabled semantic judgment with a legacy decision
• ak eval run - Execute repeated isolated evaluation trials
• ak eval triage - Compare shadow finding triage with authoritative review decisions
• ak eval triage-benchmark - Evaluate frozen finding triage against local verified labels
• ak eval ui - Report opt-in shadow UI regression signals
• ak eval ui-benchmark - Measure independently labeled UI signals offline
• ak eval workflow - Inspect a bounded scenario, fix or debug decision without executing it
• ak eval workflow-benchmark - Replay labeled workflow decisions offline