The VeriWasp GitHub App

Install the App on a repository, link the repo to one of your playtests, and every pull request gets a playtest automatically: a native check in the PR's merge box, plus a comment with the shareable evidence report. No YAML, no secrets in your repo, no pipeline to maintain.

Step-by-step setup

1. Install. Click Install on GitHub (on the homepage or your GitHub integration page). GitHub asks which account to install on and which repositories to grant: All repositories is simplest; Only select repositories works too, just remember that repos you create later need to be added by hand (GitHub → Settings → Applications → VeriWasp → Configure → Repository access).
2. Land back on VeriWasp. After the install, GitHub sends you straight back to your GitHub integration page, already linked to your VeriWasp account. If you weren't signed in, you'll go through sign-in first and the installation follows you through automatically. If the page says it's still finishing, give it a few seconds; it refreshes itself.
3. Link a repo to a playtest. On the integration page, pick the repo, pick the trigger mode (see below), and pick which of your saved playtests should run. Don't have a playtest for this repo's site yet? Use Create one for the selected repo: it opens the New Playtest form, and the new playtest links itself to the repo when you save.
4. Open a pull request. Within a couple of minutes you'll see a VeriWasp playtest check on the PR, first in progress with a live report link, then finished with the result, plus a comment with the step counts and the full report.

The two trigger modes, honestly explained

On every preview deploy (the default): tests the pull request itself. VeriWasp waits for the PR's preview deployment to go live, then runs the linked playtest against that preview URL. The code being tested is the code in the PR, before it merges. This requires your hosting provider to report deployments to GitHub, which the popular ones do automatically (see the vendor notes below). The run's report shows the exact preview URL it tested.

On every push: tests your playtest's saved target URL. The playtest runs against whatever its own target URL is serving at that moment, usually your live site or a shared staging site. The PR's code hasn't deployed anywhere yet, so this is a regression tripwire timed to development activity, not a test of the PR's diff. It's the right mode for repos without preview deployments, and it needs zero hosting setup.

If you pick preview-deploy mode but your host never reports deployments to GitHub, nothing will trigger. Switch that repo to push mode, or wire your host up using the vendor notes below.

Vendor notes for preview-deploy mode

The mechanism is GitHub's own Deployments API, so it's provider-agnostic. What matters is only that your host creates a GitHub Deployment with the preview URL and reports its status.

• Vercel: connect the repo through Vercel's GitHub integration (the normal "Import Git Repository" flow) and every PR preview reports a deployment with its URL. One thing to know: Vercel turns on Deployment Protection by default, which puts preview URLs behind a Vercel sign-in wall that the playtest browser cannot pass. Two ways through, both in the Vercel project under Settings, Deployment Protection:
  • Keep protection on (recommended): under Protection Bypass for Automation, generate a secret and paste it into the Vercel bypass token field when you link the repo on your GitHub integration page. VeriWasp stores it encrypted and sends it as the x-vercel-protection-bypass header on every request of a triggered run. Humans still hit the wall; your playtest does not.
  • Or turn it off: set Vercel Authentication to Disabled. Fine for open-source and hobby projects whose previews hold nothing sensitive.
• Netlify: works out of the box with the GitHub-connected site setup and Deploy Previews enabled (they are by default). If you use Netlify's site protection (password or Basic Auth, a paid feature), put those credentials in the linked playtest's Target requires authentication section so triggered runs can get in.
• Render: enable Pull Request Previews on the service. Render reports GitHub Deployments for previews when the repo is connected via Render's GitHub app.
• Railway, Fly.io, and others without built-in PR previews: these don't report GitHub Deployments for PRs by default. Use push mode, or create deployments from your own CI (below).
• Custom CI: if you build previews yourself, tell GitHub about them and VeriWasp follows. Two API calls after your preview is live:

# 1. Create a deployment for the PR's head SHA
DEPLOYMENT_ID=$(gh api repos/OWNER/REPO/deployments \
  -f ref="$PR_HEAD_SHA" -f environment="preview" \
  -F auto_merge=false -F required_contexts='[]' --jq .id)

# 2. Mark it successful, with the preview URL
gh api repos/OWNER/REPO/deployments/$DEPLOYMENT_ID/statuses \
  -f state=success -f environment_url="https://pr-123.preview.example.com"

When previews sit behind a login

Triggered runs inherit everything in the linked playtest's Target requires authentication section (on the playtest form, under the URL field), so any header or Basic Auth scheme your previews need is sent automatically:

• Vercel Deployment Protection: paste the bypass token when linking the repo (easiest, see the Vercel note above), or add a custom header named x-vercel-protection-bypass with the token as its value.
• Cloudflare Access in front of previews: create a service token in Cloudflare and add two custom headers, CF-Access-Client-Id and CF-Access-Client-Secret.
• Basic Auth (nginx, Netlify site protection, and similar): fill in the Basic Auth username and password fields.

All of these are stored encrypted and only ever decrypted in memory at the moment a run needs them. VeriWasp never solves or evades a login it hasn't been given credentials for: a protected preview without credentials simply fails its first step, with the sign-in wall visible in the report's first screenshot.

Making the check actually block merges

The App reports a real GitHub check, but whether a failing check prevents the merge button is your repository's own setting, like any CI check:

1. In the repo: Settings → Branches (or Rules → Rulesets on newer repos).
2. Add or edit protection for your default branch.
3. Enable Require status checks to pass before merging.
4. Search for and select VeriWasp playtest. It appears in the list once it has run at least once on the repo.

From then on, a failed playtest physically blocks merging. By default the check fails only when a step fails; the link form's gate select can tighten that to also fail on regressions since the previous run (a step that flips from passing to failing, a new console error, or a new failed network request), using the same gate rules as the CI/CD trigger. One deliberate exception: if your VeriWasp account is out of credits, the check completes as neutral, never failure, with a link to top up. A billing problem should never read as broken code, and a neutral check does not block a merge even when the check is required.

Billing, cadence, and cost control

• Every triggered run clones the linked playtest and spends credits exactly like Re-run: 1 credit per run (3 if the linked playtest has Chaos Mode on, so check that before linking a chaos run to a busy repo).
• Push mode triggers on every opened, synchronize (each push), and reopened. A rapid push session spends a credit per push.
• Preview-deploy mode triggers once per successful preview deployment.
• Repeat runs of the same flow are cheap by construction: VeriWasp reuses the previous run's element resolution wherever the page hasn't changed, so steady-state PR runs skip the AI resolution cost entirely.

Troubleshooting

• No check appears on the PR: confirm the repo is linked on your integration page, and that the App has access to the repo (GitHub → Settings → Applications → VeriWasp → Configure). Repos created after the install aren't covered unless you chose All repositories.
• Preview-deploy mode never fires: your host isn't reporting GitHub Deployments. See the vendor notes above, or switch the repo to push mode.
• The check runs but every step fails immediately, and the report's first screenshot shows a Vercel sign-in page: the preview URL is behind Vercel's Deployment Protection. Paste a bypass token when linking the repo, or disable Vercel Authentication for the project (both covered in the Vercel vendor note above), then push again.
• The check says "Playtest not run": your account was out of credits when the PR event arrived. Top up and push again.
• Every webhook GitHub sent (and whether it succeeded) is visible under the App's install on GitHub if you're debugging something deeper, and each check links its live report while the run is still going.