Evaluation method
How to Write a Bug Brief an AI Coding Agent Can Act On
A useful bug brief gives the coding task a reproducible problem and a boundary for the change. It also tells the reviewer how to judge the result. Use the template below with your own evidence; it is not a fabricated completed task or an executable Braio request.
A filled brief from a controlled browser fixture
This example uses a deliberately seeded invoice-link fixture and actual local Playwright observations. It is not a Braio agent run, a customer case or a human code-review result. The files and hashes are public so a reader can inspect the format without company data.
Scenario QA-LINK-102 clicks Open INV-102 in the invoice queue. Before the change, the browser opened Invoice INV-101 with PLN 900.00 instead of Invoice INV-102 with PLN 1,500.00. After the fixture link was corrected, the same check opened the expected invoice and amount. The report retains both observations.
| Brief field | Completed example |
|---|---|
| Scope | Synthetic invoice queue and invoice detail pages only |
| Action | Click Open INV-102 |
| Expected | Invoice INV-102; PLN 1,500.00 |
| Observed before | Invoice INV-101; PLN 900.00 |
| Acceptance | Repeat the original click and verify destination heading and amount |
| Excluded | Authentication, API authorization, bank data and a full production workflow |
Describe the failed behavior before suggesting a fix
State the affected build, environment, role and preconditions. List the smallest steps that reproduce the observation, then separate the expected behavior from what actually happened. If you have a theory about the cause, label it as a hypothesis rather than an instruction to change a particular file.
Link evidence the recipient can access. Redact tokens, credentials and personal data. If the original issue is blocked by unavailable access, ask for investigation of the blocker; do not turn an unobserved protected journey into a confirmed product bug.
Copy this brief and replace every placeholder
This is an editorial template. The placeholders are intentionally unfilled and do not describe an observed Braio or customer defect.
Title: [observable failure]
Build and environment: [revision / staging URL]
Role and preconditions: [authorized account role / test data]
Steps:
1. [first action]
2. [next action]
Expected: [agreed behavior]
Actual: [observed behavior]
Evidence: [redacted artifact and run reference]
Impact: [who is affected and how]
Allowed scope: [repository / paths / permitted operations]
Excluded actions: [production writes / external sends / other exclusions]
Hypothesis, if any: [unverified explanation]
Acceptance: [behavior that must hold after the change]
Checks: [reproduction, relevant regression and permission cases]
Reviewer and merger: [authorized human]
Retest target: [changed build and original scenario]
Open questions: [unknowns that require a decision]Make the acceptance check distinguish a fix from a workaround
A weak acceptance criterion says 'the tests pass'. A stronger criterion names the user-visible behavior and relevant boundary. For an illustrative draft-saving problem, require the permitted user to save, reload and retrieve the same draft, while another organization cannot access it. These are suggested checks, not results from an executed example.
Avoid prescribing a narrow patch that could suppress the symptom or weaken an assertion. Ask the implementer to explain the change, checks performed and remaining uncertainty. Require review if the proposed scope grows beyond the brief.
Use the same brief at review and retest
The reviewer should compare the current diff with the acceptance criterion and allowed scope. After a human merge, run the original scenario on the changed build and retain the result. If the retest is blocked, the brief remains useful but the fix is not verified.
Braio's documented defect-to-coding flow requires repository and executor configuration and permits at most two fix attempts. A human reviews and merges the PR. Confirm pilot availability and access separately; this template does not establish a completed end-to-end run.
Sources
Continue with your workflow
- Failed Test, Product Bug or Blocked Environment? A Triage Guide for AI QA
- An AI Delivery Workflow for Software Agencies: Client Request to Reviewed Result
- Independent Review for AI-Generated Code: An Evaluation Checklist
- What Counts as Evidence in AI Testing?
- AI web application testing - from scenario to retest | Braio
- AI agents for team workflows - tasks, results and approval | Braio

