Test and improve suites with ak:test

Use ak:test to turn a change or quality question into fresh validation evidence. The Skill can run unit, integration, end-to-end, build, coverage, and browser-based UI checks. It can also create a covering test suite, optimize a suite and its CI cost, or audit a suite for weak tests. Audit is report-only unless suite repair is separately authorized. The Skill's test-principles rules govern scope, layer, environment, and execution for every operation; load them before designing or changing tests.

Choose ak:test when validation is the outcome

Use ak:test when

• You need focused or broad test evidence after implementation or a bug repair.
• A project needs type, lint, build, or coverage validation before handoff.
• A website needs visual, responsive, accessibility, form, or console checks.
• You need a structured QA report, a new test suite, safer CI optimization, or an evidence-backed test audit.

Choose another workflow when

• A failing test needs root-cause diagnosis and a repair. Use ak:debug, then ak:fix.
• You want production-risk and maintainability findings from a diff. Use ak:code-review.
• You want implementation plus its default testing and review stages. Use ak:cook.
• You need to design the delivery plan rather than execute validation. Use ak:plan.

Prepare the test surface

Before you start:

• Complete Onboarding, and confirm Engineer Kit is installed for the runtime and scope you are using.
• Open the project and provide the change, requirement, failing area, or test scope that should determine the commands.
• Keep the project's test runner, dependencies, fixtures, environment variables, and build tooling available.
• For UI testing, provide the URL and start the required application yourself or explicitly authorize the workflow that starts it.
• For protected UI routes, choose project-native test authentication or approve use of the real Chrome profile and complete any required manual login.

Select the operation

If you provide a scope, the Skill proceeds with relevant code validation. With no arguments, it asks which operation to run. Use ui <url> to select the browser workflow directly, or start with create, optimize, or audit to work on the test suite itself. Any suite change, optimize or audit proposal, or QA report must stay consistent with the test-principles rules.

/ak:test "Validate the session expiry change: run typecheck, the focused session tests, affected integration tests, and coverage for the changed module."

/ak:test ui http://localhost:3000/settings

/ak:test "Validate the session expiry change: run typecheck, the focused session tests, affected integration tests, and coverage for the changed module."

/ak:test ui http://localhost:3000/settings

$ak:test "Validate the session expiry change: run typecheck, the focused session tests, affected integration tests, and coverage for the changed module."

$ak:test ui http://localhost:3000/settings

For create, optimize, and audit, add --advice for kongming advisory checkpoints or --interview to approve each proposed change group before edits. --ultra runs five read-only analysis candidates: create and optimize select one winning design unchanged, while audit returns the evidence-validated, deduplicated union of findings. Five initial candidate slots are dispatched in one parallel wave; failed slots are retried once. If parallel read-only dispatch is unavailable or fewer than five results are usable after the retry, the run fails closed. Without --interview, create and optimize apply in-scope changes directly and report the full change list; omitting the flag does not authorize audit repairs or test retirement.

/ak:test create "Cover the account recovery workflow" --interview
/ak:test optimize "Reduce pull-request CI time" --advice
/ak:test audit "Check skipped and ineffective authorization tests" --ultra

Project-defined commands and thresholds take priority. When the project does not define a coverage threshold, the Skill reports actual coverage and the critical behaviors still missing tests; it does not invent a percentage gate. Reuse adequate existing tests; a covered refactor can add zero new tests.

Before adding tests or running broad checks, name the accepted contract and its authority, affected risk, suitable existing check, and remaining evidence gap. An inferred expectation is a hypothesis until checked against that authority. Admit a new test only for a concrete protection gap; demonstrate its intended failure when safe and practical, or report that proof as unavailable.

Combining, lowering the layer of, or retiring tests requires lifecycle review of the protected contracts, regression history, measured cost, and remaining protection. Sampled mutation-kill overlap or unchanged line coverage alone does not justify retirement. Review the remaining suite jointly, preserve required tests and mandatory gates, and obtain suite-repair authority for retirement. After authorized changes, rerun affected contracts and the same valid sampled faults against the remaining suite, followed by required gates.

Use impact selection and optional semantic modes

For a repository with a trusted test-impact manifest, optimize uses ak test optimize --base <ref> --json to plan a ranked, deterministic selection of check groups with selected and skipped reasons. Planning is read-only. --run executes the selected repository-owned commands, which is not a sandbox, and --output <new-file> keeps a local result. Uncertainty broadens selection toward the declared full suite, mandatory groups and policy checks stay required, and unresolved policy mappings block execution. This path contacts no semantic provider.

Semantic modes are opt-in only, and the Skill never infers consent from a key in the environment. With an explicit test-impact consumer grant, --semantic shadow leaves the effective baseline unchanged, and --semantic advisory can only add checks or broaden selection. Granted semantic modes can send compact facts to an external provider and incur charges. ak test classify-e2e needs its own grant and never changes E2E scheduling or the class manifest. For UI testing, a requested ak eval ui shadow experiment with an explicit ui-regression grant can add regression signals from already-observed evidence; it never turns a failure into a pass or changes the UI test result.

Optional agentic E2E through tester-army/e2e is only for projects already using it or when you explicitly request setup. This does not authorize installation, provider access, model calls, or telemetry; ordinary native UI checks need no model. Check the installed engine's capabilities before running it. Its cache replays actions, not passes: rerun exact outcome assertions on the current app. --strict-cache alone is not a no-model guarantee. If model calls are forbidden, use independently verified enforcement for the selected version or stay on the native route. Report actual replay/live execution and unfinished exploration; no findings or a cache hit does not verify unexercised release claims.

Understand what happens during a run

The default and ui operations follow this validation flow:

1. The Skill determines scope. It maps the accepted contract, authority, and risk to the smallest useful check at the lowest reliable layer, and names missing evidence before adding work.
2. The Skill runs the narrowest relevant check first. Type, syntax, lint, or build checks run when they establish the affected contract; repository-required pre-flight gates remain mandatory.
3. The Skill executes the selected suites. It starts focused, expands to integration or end-to-end coverage when the blast radius requires it, and preserves skipped, flaky, slow, and failed tests.
4. The Skill measures coverage when relevant. It reports line, branch, and function coverage, then identifies uncovered files, error handlers, and edge-case branches instead of relying on one percentage.
5. The Skill verifies the build. It runs the project-appropriate production build when build compatibility belongs to the requested scope.
6. The Skill exercises the UI when selected. It sets a narrow charter with routes, expected behavior, account/data authority, and mandatory checks. It uses an isolated browser, authorized profile-aware Chrome, or native browser tests, sequentially by default; only independent, isolated targets justify delegation. Browser checks establish browser-observable scope, not backend coverage. Assert exact identities, permissions, counts, side effects, and any specifically claimed control/path; an alternate route cannot rescue a broken control. Capture screenshots only when relevant to the visual claim.
7. The Skill produces the report. It organizes results, failures, coverage, build status, critical issues, recommendations, and unresolved questions in the configured report location.

Keep test effects controlled

• Test commands may create coverage files, screenshots, build output, temporary data, or browser state. State allowed artifact locations, execution/retry budgets, and cleanup requirements; clean only resources owned by this run.
• Mutation proof is selective, not a routine execution or CI requirement. Use bounded fault injections only in owned disposable copies, never the active worktree. Record source identity, restore the copy exactly even on failure, verify restoration, and rerun restored behavior. Only a target assertion failure is a semantic kill; compilation, setup, infrastructure, or provider failures are invalid or inconclusive evidence, not kills.
• Run suites against a safe test environment. UI flows and end-to-end tests can submit forms or mutate data if the application behaves that way.
• Real Chrome profile testing can access the user's authenticated session. Use it only when that state is required and bind to the exact profile-aware tab.
• External URLs, hosted browsers, performance services, or production-like environments can incur provider or operational cost and need separate authority.
• Default and ui operations report product defects without implementing fixes. Audit repairs require separate suite-repair authority; product root-cause work still belongs in ak:debug and ak:fix.

Verify the report

A complete default or ui result should give you:

• The exact scope and every command executed.
• Accepted contract and authority, source/input fingerprints before and after execution, runner/configuration, environment, and linked receipts or artifacts.
• Totals for passed, failed, and skipped tests, plus measured duration; retain every failed attempt and its raw status even when a retry passes.
• Complete failed-test names, essential error output, and relevant stack traces.
• Coverage values and concrete gaps when coverage was requested.
• Build status, warnings, and dependency issues when build verification applied.
• UI pages and flows tested, screenshot paths, console findings, viewports, and any skipped browser checks.
• Critical issues and prioritized recommendations.
• Environment, authentication, or evidence limitations listed explicitly.
• Applied versus proposed suite changes, their authority, and remaining protection. Compare costs only under matched source tasks, environment, toolchain, and cache conditions: include observed setup, generation/review, runner/model calls, tokens, CI minutes, and device resources where available. Unknown metrics stay unknown; elapsed time is not billed CI minutes, and fewer tests or tokens alone do not establish savings.

Treat the run as passing only when the requested checks report zero failures, the build exits successfully when included, and project thresholds are met. A partial or skipped check proves only the surface actually executed.

A skipped or unavailable mandatory gate means insufficient evidence. Reuse receipts only when their scope, fresh source, relevant dirty/untracked inputs, configuration, and environment match; a commit SHA alone is insufficient. Invalid, stale, or foreign-scope receipts require rerunning the necessary checks, not blindly repeating the whole suite. Changed or unverified final inputs prevent a final-source pass claim; report the tested state and remaining gap.

Troubleshoot or continue

Hand failures to ak:debug and ak:fix, or continue with ak:code-review after the requested validation passes.

Know the current limits

• The Skill is limited to the test runners, browsers, credentials, URLs, and environments available in the current session.
• Coverage percentages do not prove behavior outside the assertions or runtime conditions exercised.
• Visual screenshots do not replace interaction, accessibility, console, or repeatable project-native tests.
• Cursor slash invocation is user-verified evidence, not proof of full runtime parity.
• Available operations depend on the installed Engineer Kit release.