Skip to main content

Ask AI About a Run

A report tells you what happened: which steps passed, which failed, what issues were found. It doesn't tell you why, or what to actually do about it, especially if you're not used to reading raw error messages or accessibility violation codes. Once a run reaches completed or failed status, its detail page shows an Ask AI about this run box that answers exactly that kind of question.

What it is​

A single-question box, not a persistent conversation. Type a question, click Ask, and get an answer grounded in that specific run's own data: its target URL, every step's action type, description, and outcome, any console or network errors captured, and every issue found. Each question is independent; the model has no memory of a question you asked a moment ago in the same box, so if you want to reference something from an earlier answer, include it in your next question.

Some useful things to ask:

  • "Why did step 3 fail?"
  • "How do I fix the accessibility issue about contrast?"
  • "Is this a real bug in my app, or a problem with how the test step was written?"
  • "What's the most likely cause of the failed network request?"

Where it lives​

Only on the private, authenticated run detail page, for runs that have reached completed or failed. It's deliberately not on the public shareable report: unlike the report itself, answering a question costs a real API call, and a public, unauthenticated surface with no rate limiting tied to an account would be a straightforward way to run up real cost against your account for free.

Cost​

Free for now. Answering a question doesn't deduct a credit or charge anything, unlike creating and running a test itself. This may change once there's more usage data to inform pricing it sensibly; the only limit today is a light rate limit per account (generous enough for normal debugging use, tight enough to prevent a scripted loop from running up cost) since a question still triggers a real, real-money API call to a large language model even though it isn't billed to you today.

Accuracy​

Answers are grounded in the run's actual captured data and instructed to say so plainly rather than guess when the data doesn't contain enough to answer confidently, but this is still a language model's interpretation of that data, not a deterministic calculation. Treat answers as a well-informed starting point for debugging, the same way you'd treat a colleague's first guess after skimming the same report, not as a certain diagnosis.

Model​

Uses a stronger, more expensive model than the one that decides which element to click during a run (see Step Types & AI Resolution). Element resolution is a narrow "pick the best match" task; explaining a failure or suggesting a fix needs actual reasoning, which is worth the extra cost here specifically.