Issues & Accessibility Scanning

Beyond a simple pass/fail per step, every run produces a list of issues: plain-English findings meant to read like something a thorough human tester would write up, each tagged with a category and a severity. This list appears on both the private run detail page and the public shareable report, in an "Issues found" section, and is counted in the summary stats at the top of the report (Steps / Passed / Failed / Issues found).

There are four issue categories, generated by several different mechanisms.

Failed steps (category: bug)

Every step that fails generates a bug issue at high severity automatically, with a summary in the form:

Step 3 (click) failed: on my credit balance in the top nav

and a detail line with the exact underlying error (almost always could not find an element matching "..."; see Step Types & AI Resolution for why a step fails to resolve). This is a direct, mechanical translation of a failed step into an issue. There's no additional judgment call being made here beyond "this step didn't succeed."

If a click or type step failed after being automatically retried (see below), the summary says so explicitly:

Step 3 (click) failed after 2 retries: on my credit balance in the top nav

Automatic retries and "passed on retry" (category: usability)

click and type steps that fail are automatically retried, up to twice (three attempts total), before being reported as genuinely failed. Element-not-found and action-timeout failures are usually a transient timing issue (the page hadn't finished rendering yet, a brief network blip), not a real bug, and reporting one as "your signup is broken" when it works fine a moment later would erode trust in the tool more than a false negative would.

Two things are deliberately excluded from this:

• assert steps are never retried, no matter which assert kind. A failed assertion is the actual finding a step was written to check for, not flakiness to paper over.
• Chaos Mode disables retries entirely. Chaos Mode's whole purpose is surfacing exactly this kind of flakiness under adverse conditions (throttled network, injected noise), so automatically retrying past it would defeat the point. A chaos-mode step fails on its first miss, same as before this existed.

Retries share the step's existing execution timeout budget rather than extending it, so they can never blow out a run's total wall-clock time beyond what a single slow, ultimately-failing step could already take.

If a step needed a retry but passed on a later attempt, that's still surfaced, just as a low-severity usability finding rather than silently hidden:

Step 4 (the checkout button) passed on retry 1. This step didn't work
on the first attempt.

The reasoning: a step that only succeeds after a retry is itself a signal that the target is a little slow or flaky, worth knowing about even though the step ultimately passed. Both the run detail page and the public report also show a small "retry N" badge directly on any step that needed one.

Console errors (category: bug)

If the browser's JavaScript console logged any errors at all during the run (across every step, combined), a single bug issue at medium severity is added summarizing the count:

3 browser console error(s) logged during the test

This is a count, not a full dump of each individual error's message and stack trace on the report page itself. It exists to flag that something threw client-side, prompting you to open your browser's own devtools console against the same flow if you need the specific error text and stack trace.

Failed network requests (category: bug)

Separately from console errors, VeriWasp also watches the browser's network traffic during every step and flags failed requests as their own bug issue, at high severity (one severity level above a console error, since a failed API call or a failed page navigation is almost always a real, functional problem rather than a benign warning):

2 failed network request(s) during the test

A request is flagged as failed if either the server responded with a 4xx or 5xx status, or the request failed outright at the network level (DNS failure, connection refused, timed out) and wasn't simply canceled by navigating away.

Only same-origin page loads and API calls are flagged (technically: requests of type Document, XHR, or Fetch). Sub-resource requests like images, stylesheets, fonts, and third-party analytics/ad scripts are deliberately excluded, since those fail constantly on real websites for reasons that have nothing to do with the flow being tested, and would otherwise drown out the failures that actually matter.

This exists specifically to catch the class of bug that throws no console error at all: a "Save" button whose click handler calls an API, gets back a 500, and silently does nothing, no JavaScript exception, no console output, just a button that appears to do nothing when clicked. Pure console-error monitoring misses this entirely; network-level monitoring catches it.

Slow steps (category: usability)

Any step whose execution took longer than 3 seconds is flagged as a usability issue at medium severity:

Step 4 (the checkout button) took 4.2s. Slow interactions like this
frustrate users on mobile or a weak connection.

wait steps are deliberately excluded from this check. A wait step's duration is intentional (you asked it to pause), not a symptom of the page being slow, so it would be misleading to flag it as a usability problem.

Accessibility scan (category: a11y)

At the very end of a successful run (specifically, only if at least one navigate step actually succeeded during the run; a run that never successfully loaded the target page at all skips this step, since there's no page state left to scan), VeriWasp injects axe-core, the same open-source accessibility testing engine used by many commercial and open tools, into the page and runs a full WCAG scan against whatever page the browser was on when the run finished.

Each violation axe-core reports becomes one a11y issue, with:

• A summary in the form <rule description> (<N> elements), for example "Elements must meet minimum color contrast ratio thresholds (11 elements)."

• A detail line with axe-core's fuller description of the rule being violated.

• A severity derived from axe-core's own impact rating for that violation:

  axe-core impact	VeriWasp severity
  critical or serious	high
  moderate	medium
  minor (or anything else)	low

Because this scan only runs against the page state at the end of the run, it reflects whatever page your last step left the browser on. If your scenario ends on a confirmation page, that's what gets scanned, not the homepage or any intermediate page visited earlier in the run. If you specifically want accessibility findings for a particular page, make sure your scenario's last step (or an assert immediately before the end) leaves the browser there.

Reading severities

Severities (high, medium, low) are rendered with color-coded badges wherever issues are listed:

• High: red badge. Failed steps and serious/critical accessibility violations.
• Medium: amber badge. Console errors and slow steps, plus moderate accessibility violations.
• Low (and anything else): neutral badge. Minor accessibility findings.

There's no numeric score or overall grade computed from these. The report's top-line stats are simple counts (total steps, passed, failed, and total issues found across all four categories combined), and the issue list itself is where the actual substance lives.