QA checks | AOSplane | Documentation

QA checks are commands, such as your tests, that prove a change works. When you start a session with Loop until checks pass or Plan first, QA runs every check on each try, after the agent has built and reviewed the change. A check passes when its command exits 0. If one fails, the work goes back to the agent; once all pass, a draft pull request opens.

Owners and Admins add and change QA checks. Recording checks on video needs the Verified PRs add-on, sold on the Starter, Team and Enterprise plans.

Add QA checks to a repository

Open Repository setup

In your project, choose Settings, then Repository setup, then Configure on the repository’s card.

Open QA checks

Choose the QA checks tab.

Let AOSplane suggest checks

Choose Detect from repository. Under Suggested from your default branch, choose Add all, or Add next to the ones you want.

Or add one yourself

Enter a Name and a command such as pnpm run test, then choose Add check.

Save

Choose Save settings. Suggestions aren’t saved until you do.

QA checks tab with a list of named check commands, the Add check and Detect from repository buttons, and the Record browser checks on video card — The QA checks tab in Repository setup

Run a session against them

Open the composer

Choose New session and pick the repository.

Pick a start mode

Open Session settings, choose Start, then Loop until checks pass (or Plan first, then wait for approval).

Set Max tries

Pick 1 to 5 tries. The default is 3.

Follow QA

Send the task. The session’s Recording tab ticks each acceptance criterion and repository check that passed, crosses the ones that failed, and counts how many are proven.

Session Recording tab showing QA round 1 with 6 of 6 proven, the recorded video, and the Acceptance criteria and Repository checks lists with every check passed — QA results in a session's Recording tab

Record browser checks on video

With Verified PRs, QA records each Playwright check it runs and the pull request links every video. The Record browser checks on video card in the QA checks tab walks you through it, with a button for each step:

A QA check runs Playwright, for example pnpm exec playwright test.
Your playwright.config.ts saves videos where QA collects them. Choose Copy config for the snippet.
Playwright uses the sandbox’s Chromium. Choose Add variable to add CHROMIUM_PATH=/usr/bin/chromium.
The setup script installs Playwright’s video encoder. Choose Add to setup script to add npx playwright install ffmpeg.
If the network is limited, choose Allow hosts so it can reach Playwright’s downloads.

Then choose Save settings. Don’t have the add-on? Choose Add Verified PRs on the card.

Good to know

Each check runs in the repository’s folder with bash -eo pipefail, after the setup script, with the repository’s variables and secrets.
Every check runs even after one fails, and each has its own timeout.
A repository holds up to 20 checks. Names must be unique; a blank name uses the command.
Detect from repository reads package.json scripts, go.mod, Playwright and pytest configuration at the repository root.
A session keeps the checks it started with. Changes apply to new sessions.
Each try is one round of build, review and QA. If the last try still fails, the session stops without a pull request.
Plan first adds the approved Spec’s acceptance criteria to the checks. See Product manager.
Verified PRs costs $29 a month per organization with 300 recorded QA minutes included, then$ 0.10 a minute. A recorded minute is the run time of a check that saved a video, rounded up. Videos are kept for 30 days. See Billing.