Visual QA Loops

Position this module against the previous one. Module 4 was about parity: one page, one reference, one convergent loop. Visual QA is the wider net — every route the prototype has, every state that matters, every breakpoint, checked the same way every time. The skills overlap heavily; what changes is scale and repeatability.

The core claim of the module is in the first bullet: a generated interface can compile, load, and pass every functional test while still failing the design. Spacing rhythm, density, contrast, the empty state nobody designed, the layout that holds at 1440 pixels and collapses at 390 — none of that shows up in a type check, and most of it slides past code review because the diff looks reasonable. The only honest check is looking at what a user would actually see, and the only scalable way to look is screenshots an agent captures and compares on your behalf.

Flag the boundary early because it shapes everything else: visual QA with an agent is a review activity, not a license to change the UI. The agent captures, compares, and reports. A human decides what counts as a defect, what counts as taste, and what counts as done. That gate is not overhead; it is the difference between a QA loop and an agent quietly redesigning the prototype one confident finding at a time.

The failure modes worth naming concretely: a page that uses the right component library but misses the intended hierarchy; copy that matches the brief while the rhythm is broken; a layout that holds at 1440 pixels and falls apart at 390; an empty state that was never designed so the agent invented one. None of these surface in a type check or a unit test, and they rarely surface in code review either, because the code is plausible. They surface in front of stakeholders, which is the most expensive place to find them.

The second point is where most teams under-invest. One desktop screenshot of the happy path is the default evidence, and it is nowhere near enough. The bugs live in the combinations — the error state at phone width, the dense list at tablet width, the dark theme nobody re-checked after the token change.

The last bullet explains why this pairs so well with agents specifically. Capturing the same routes at the same three widths, re-running the same accessibility scan, and re-reading the same checklist after every fix pass is exactly the kind of repetitive, evidence-heavy work humans do badly on the fourth iteration and agents do identically on the fortieth. The human contribution is not the repetition; it is deciding what the evidence means.

The matrix is a design artifact, not test infrastructure. Writing it forces the questions a prototype review needs answered anyway: which routes actually matter, which states exist, and what each page is for. Keep it in the repository as a manifest file so it is reviewed like code and grows with the prototype — a page that is not in the manifest is a page that will never be checked.

On breakpoints: the specific numbers matter less than fixing them and never varying them. This school's own captures use 390, 768, and 1440 pixels; add 1024 if tablets matter to your audience. If every run uses slightly different widths, findings stop being comparable across runs and the whole baseline idea collapses.

States are where prototypes are weakest, because prototypes are usually built against the happy path with friendly demo data. Empty, loading, error, and dense are the four to insist on; selected, expanded, and disabled follow for interactive surfaces. The intent line deserves its own emphasis: one sentence per route describing the user job. It is what lets a compare agent — or a human — distinguish a difference that breaks the page's purpose from a difference that is merely a difference.

Three kinds of baseline cover most situations, and they answer different questions. A reference baseline — design exports or the screenshots you held parity against in Module 4 — answers does the build still match the design. An approved baseline — the captures from the last build a human signed off — answers did anything change that nobody intended; this is the regression question, and it is the one that matters most once the prototype is moving fast. The intent baseline — the design-system rules, the tokens, and the per-route intent line — answers does the page still do its job, and it catches the failures the other two cannot, because it is not pinned to any particular set of pixels.

The mechanics matter as much as the concept. Baselines live on disk, next to a manifest recording route, viewport, and capture date, so the evidence survives the session and the next review compares against the same thing rather than a memory of it. Saving captures to disk also avoids the token cost of pasting full-page screenshots into a conversation, which practitioner reports have measured in the six figures for a single image.

The last bullet is the discipline point: the baseline only moves when a human promotes it at the gate. If the agent updates the baseline whenever a sweep passes, regressions get silently absorbed into the new normal and the baseline stops meaning anything.

Walk the diagram in order and name who owns each step. Capture is agent-run: screenshots at the fixed widths, the states from the manifest, an accessibility snapshot, and automated check output, all saved to disk with a manifest. Compare is agent-run but bounded — observable differences only, judged against the baseline, the tokens, and the intent line, never against the agent's own taste. Findings is the report: every finding carries a severity, the evidence it rests on, the likely cause, and a proposed fix, with verified and candidate findings clearly separated. The approval gate is human and is where this module's title earns its keep — the human decides which findings are defects, which are taste, and what done means for this prototype. The fix pass is agent-run and scoped to one concern at a time, and recapture re-runs the same commands at the same widths so the team knows the fix did what was asked and nothing else.

Two failure modes the order prevents. Teams that skip the evidence step ask the agent to critique from code, and it obliges with confident guesses about rendering it never saw. Teams that skip the gate let the agent fix its own findings immediately, and taste decisions get made silently inside a diff.

Point at the dashed line: the loop runs until no P0 or P1 remains, and the new captures become the baseline only when the human at the gate says so.

The capture layer is no longer something you invent. Microsoft's Playwright MCP server gives an agent browser navigation, viewport resizing, screenshots, and — often more usefully — a structured accessibility snapshot of the page, which answers layout-order and labelling questions at a fraction of the token cost of an image. Google's Chrome DevTools MCP server adds console messages, network requests, and performance traces for the cases where a page looks wrong for non-visual reasons, such as a font that never loaded. A small capture script in the repository wins when you want the identical evidence set every run; driving the MCP tools interactively wins when the agent needs to poke at one state. As of June 2026, both servers are the standard way agents reach a real browser.

The sweep pattern from this school's regression-sweep workflow scales the single review by separating orchestration from judgment: a script walks the manifest, captures everything, then fans out one narrow compare agent per page. Each compare agent sees only its own pair of captures and its route's intent line, which keeps its findings grounded rather than averaged across the whole product. The orchestration merges and ranks the results, so the human reads one report.

The most useful signal a sweep produces is repetition. If eleven pages report the same loosened card padding, the cause is almost certainly one shared token or component, and one fix clears eleven findings. The merged report should group repeated findings by likely shared cause for exactly that reason.

The rubric exists to stop two failure modes: polishing details while the product task is still broken, and treating every difference as equally urgent so nothing gets prioritised at all. Assigning severity by user impact rather than fix effort is the discipline that keeps it honest — a one-line layout fix can be a P1 and a finding that needs two new assets can be a P2, and conflating effort with severity is how trivial-but-cosmetic fixes crowd out cheap-but-important ones.

Give the levels concrete faces from real reviews. A P0 is a save button that falls below an overflowing panel at 390 pixels, or a chart legend rendering white on white after a theme variable rename — both real findings from a sweep on a fourteen-page product, both invisible to functional tests. A P1 is a sticky header whose stacked navigation pushes the page title toward the fold on phones. A P2 is navigation that exposes no current-page state. A P3 is whether a display type scale earns its cost on small screens when the design system explicitly specifies it.

Reading findings also means separating real regressions from acceptable change. Some differences are the design behaving exactly as specified, and some are deliberate improvements that simply differ from the baseline. The report should mark each finding as verified or candidate, and as objective mismatch or design judgment — a report containing only confirmable defects has usually been filtered to look objective, and a report that is mostly taste has not done the comparison work.

This quality bar matters most when an agent will implement the fixes. If the report says spacing feels off, the next agent may redesign the whole page; if it says the header consumes a third of the 390-pixel viewport because nine links wrap into four rows, the fix can stay narrow and the recapture can prove it worked. Precision in the finding is what keeps the fix pass scoped.

The evidence rule is the single fastest improvement you can make to report quality: every finding names the file, the viewport, and where relevant the axe rule it rests on. Findings that cannot name evidence split cleanly into two groups — design judgments, which are welcome as long as they are labelled as judgments and routed to a human, and guesses, which go back to the agent with a request to verify or drop.

The fourth row is about the report as a whole rather than individual findings. A sweep that returns two hundred undifferentiated pixel notes has produced a diff log, not a decision artifact. The strong version leads with blockers, groups repeated findings by likely shared cause, and gives the polish items owners. The examples in this table are drawn from this school's own published review of its article page and from the regression-sweep workflow's case studies; they are real shapes, not invented ones.

The argument for folding accessibility into the visual sweep rather than treating it as a later stage: most of the failures automated tools can catch are visual and structural anyway — low contrast, missing focus states, unlabelled controls, colour used as the only signal, heading levels that skip. The evidence is already being captured for the visual review; running axe against the same routes costs one extra command and the findings translate directly into the same P0 to P3 format, so the gate sees one report instead of two.

Tooling, briefly and as of June 2026: axe-core is the standard rules engine, runnable from the command line with @axe-core/cli or inside a Playwright spec with @axe-core/playwright, which matters because the Playwright route can test states behind interactions, not just initial loads. Lighthouse CI wraps Lighthouse runs in assertions and budgets so a score regression can fail a branch the way a broken test does, and pa11y-ci is a lighter URL-sweep runner for CI. All of them emit JSON an agent can read and prioritise.

The caveat has to be stated as firmly as the value: automated checks catch a meaningful share of issues and prove nothing on their own. They cannot tell you whether the focus order makes sense, whether the alt text is useful rather than merely present, or whether the page works with a screen reader in practice. A clean axe report is an entry condition for review, not a result.

The placement question is where prototype teams most often get this wrong. Visual QA borrowed from production habits gets scheduled at the end — a big sweep before the stakeholder review — which means problems are found when the prototype is largest, the deadline is closest, and every fix risks disturbing something else. Inside a prototyping loop the economics invert: captures are cheap, the agent is already iterating, and a small sweep after each milestone catches drift while it is one fix instead of forty.

Connect this to the loop the course has been building since Module 1. The QA sweep is not a new ceremony bolted onto the side; it is the critique step made repeatable and evidence-based. Findings from one sweep become the critique that steers the next agent run, and findings that keep recurring — the same spacing drift, the same forgotten focus state — are a signal that a rule belongs in the harness or the design system rather than in another round of feedback.

The last bullet keeps the loop honest about what a prototype is. Module 1 set fidelity decisions per layer: visual, data, interaction, content. The QA matrix should test the claims the prototype actually makes and not the ones it explicitly faked. Filing P2s against placeholder data the team agreed to fake wastes the gate's attention and trains people to ignore the report.

This trace comes from the regression-sweep workflow published on this school's site; the numbers are from that case study, not a controlled benchmark, and are worth presenting that way. The setup is the situation this module has been describing: many small changes merged over nine weeks, nothing obviously broken, and a release date close enough that a manual page-by-page review was not going to happen.

The two P0s are the instructive part. The save button below the overflowing panel at 390 pixels is a pure breakpoint regression — the desktop layout was fine, the functional tests passed because the button existed and worked, and only a capture at phone width made the problem visible. The white-on-white chart legend is the other classic shape: a theme variable rename that no individual diff made look dangerous. Neither finding required sophisticated judgment to act on once it existed; both required evidence nobody was collecting.

The resolution is as important as the detection. The team did not fix all 31 findings; they fixed the two blockers and the four P1s that traced to shared causes, recaptured the affected pages with the same commands, and shipped. The sixteen P2s became a backlog with owners instead of a vague sense that the release felt rough, and the six P3s went to a designer as decisions. That is what a working gate looks like: the sweep narrowed the decision; humans made it.

The limits split into what the loop cannot see and what it cannot decide. It cannot see pages that are not in the manifest, states that need login or data setup nobody scripted, or anything about whether users will actually understand the flow — visual QA is not usability testing and does not substitute for it. It cannot decide whether the baseline was good in the first place, whether a P3 judgment call becomes a change, or whether the prototype ships with known P2 debt. Those are design and product decisions, and the loop's job is to put them in front of the person who owns them with evidence attached.

The tooling failure modes are worth a minute because a workflow that does not name them ends up debugging its own evidence. Full-page captures of long pages produce artifacts that look like defects: sticky headers repeated mid-scroll, lazy-loaded images that never fired, animations frozen mid-state, fonts captured before they swapped. Wait for network idle, keep the capture deterministic, and treat any finding that only appears in a long stitched capture with suspicion until a viewport-sized shot confirms it. And keep evidence on disk rather than pasted into the conversation — the token cost of inline full-page screenshots is real and it crowds out the context the agent needs for the actual review.

The organisational anti-pattern outranks all the technical ones: letting the agent apply fixes the moment it produces findings. The entire value of the loop is that a human reads the prioritized report before the UI changes.

The deliverable is a manifest file and a one-page packet skeleton, not a working sweep — the capture script and the compare agents can come later, and for a small prototype the first run can even be manual. What matters is that the decisions are made and written down: which routes, which states, which widths, what the comparison is against, and who decides what counts as done.

The constraint to enforce is the focus on three routes. Most people's first instinct is to list everything, which produces a matrix nobody will ever run. Three routes crossed with four states and three widths is thirty-six captures — already a meaningful sweep and still small enough to read the results in one sitting. The matrix can grow once the loop has run a few times.

The last bullet is the one participants skip and the one the module is named for. Naming the person at the gate — even when it is themselves — forces the question of what done means for this prototype: which severities block the demo, which get logged, and who decides the taste calls. If the exercise is run in a group, comparing intent lines is the most useful discussion: two people writing intent lines for the same route usually discover they disagree about what the page is for, which is a finding in itself.

Recap by connecting the bullets back to the loop diagram rather than re-listing them. The matrix decides what evidence exists; the baseline decides what the evidence is compared against; the sweep produces findings that are only as useful as the evidence they name; the rubric turns those findings into decisions; and the gate keeps the decisions human. Each piece on its own is small — a JSON file, a folder of screenshots, a severity table — and together they are what makes visual quality checkable instead of vibes.

It is worth restating the module's one cultural rule because the next module will stress it: the agent reports, the human decides. In the sprint, time pressure is exactly the condition under which teams are tempted to let the agent fix its own findings and to let a clean sweep stand in for a decision about readiness. The habits built here are what hold under that pressure.

Preview Module 6 concretely. It runs the whole course at full speed: planning a prototype sprint as a sequence of bounded agent runs with gates, a mid-sprint critique to catch drift while it is cheap, testing with users or stakeholders, and the honest handoff that distinguishes built, faked, and unknown. The QA matrix from this module's exercise becomes part of the sprint's definition of done, and the sweep report becomes one of the artifacts the handoff is built from.