System Audits and Drift Detection

Modules 1 to 3 built the system's contract: tokens that carry intent, a DESIGN.md the agent loads every session, and review gates that keep component taste deliberate. This module is about the gap between that contract and what actually ships. Drift is not a failure of discipline; it is what happens when a product is built faster than any one person can watch it. A one-off colour during a deadline, a copied component with a tweaked radius, a focus state that never got ported — none of these feels like a problem in the moment, which is exactly why nobody catches them.

The claim of this module is narrow and practical: agents change the economics of finding drift, not the politics of fixing it. One agent can read one component folder with full attention; sixty agents can read sixty folders in parallel. What used to be a heroic one-off project — weeks of designer and engineer reading time, ending in a slide deck nobody reruns — becomes something a team runs quarterly or after every rebrand.

Set the tone for the output early. The point of an audit is not a guilt document that lists everything wrong with the system. It is a decision artifact: ranked findings, each with evidence, each with a likely owner, feeding a fix plan the team can actually schedule. Everything in this module is in service of that artifact.

The examples on the slide are deliberately mundane because that is what real drift looks like. Nobody decides to fork the design system; they decide to ship the feature on Friday. The copied card with a slightly different radius, the warning badge that is orange on one page and red on another, the disabled button that looks like a secondary button — each was a local decision that made sense under pressure.

Push past the aesthetic framing. The deeper cost is meaning: a status colour that differs across pages teaches users two different languages for the same state. A copied modal that omits focus management turns a shortcut into an accessibility regression. A button fork with a different touch target changes how reliable the product feels on mobile. Drift is a product-quality problem wearing a consistency costume.

This is also why hand audits fail. Reading every component folder, every consuming route, and every place a hex value was typed instead of a token takes weeks, so most teams do it once, produce a deck, and never repeat it. The drift continues the day after the deck is presented. The rest of this module is about making the audit cheap enough to repeat.

The table compresses two framings from the school's audit article: the audit dimensions you scope, and the four layers of drift — meaning, behaviour, implementation, presentation — that explain why the same visible inconsistency can have different causes and different owners. A file search can find raw hex values but cannot tell whether the UI still communicates the right priority; a screenshot review can spot inconsistent cards but miss that two implementations use different components underneath. You need both code-level and rendered-output checks.

Walk one example through the layers. Two status chips look different. If the difference is that escalated renders red on one page and orange on another, that is meaning drift and it is severe. If one chip lacks a disabled state, that is behaviour drift. If one is a local copy that bypasses the shared StatusChip, that is implementation drift, and the fix is consolidation. If only the padding differs, that is presentation drift, and it might even be intentional in a dense table.

The dimensions are also what stop the audit collapsing into vague commentary. 'Buttons are inconsistent' is not a finding. 'The escalated chip uses a raw red on the project detail route while the dashboard uses the semantic warning token' names the dimension, the evidence, and the likely fix path.

Scope is where audits are won or lost. 'Audit our app design and make it consistent' produces vague commentary; 'audit status chips across dashboard, project detail, mobile queue, and settings' produces findings. The scope should name the component family or surface, the routes where it appears, the source of truth to compare against, the screenshots or states to inspect, and the allowed output. The agent finds and classifies drift; it does not silently redesign the system while it audits.

The three scoping axes on the slide map to who acts on the result. Per component family is the strongest default because one family used in many places concentrates the most reusable findings. Per route or package suits release-driven work and marketing surfaces. Per team matters in larger organisations because a finding without a plausible owner just sits in the report.

The last bullet is the one teams skip: write the reference down before any agent runs. The token set, the spacing scale, the documented variants, and — critically — the known intentional exceptions. If a designer deliberately approved a compact chip variant for dense tables, say so in the scope, or the agent will dutifully report it as drift. Ambiguity in the reference becomes noise in the findings.

Walk the diagram left to right. The scope and reference are human work — the previous slide. The fan-out is an orchestration script the agent writes: it enumerates the audit units with a glob so the list stays accurate as the system grows, builds one audit packet per unit, and dispatches one subagent per component folder or page, in batches. In Claude Code this runs as a dynamic workflow: the intermediate findings live in script variables rather than the main context, up to sixteen agents run concurrently, and an interrupted run resumes rather than starting over. As of June 2026 a single run can dispatch up to a thousand agents, which is more than any design system needs.

The per-unit auditor is deliberately narrow: it reads one folder, compares against the reference it was handed, and returns structured findings — file, line, category, severity, evidence, suggested fix — as JSON. Narrow agents produce findings that merge cleanly; broad agents produce essays that do not. Defining the auditor once in .claude/agents/ keeps behaviour consistent across runs.

The right side is where humans return. The merged report ranks findings and surfaces repeats; the review gate triages drift versus intentional variation, approves mechanical fixes, and pushes recurring findings into the harness. The dashed line back to scope is the part that makes this a practice rather than a project: save the orchestration to .claude/workflows/ and rerun it, so each report becomes a point on a trend line.

Evidence is the difference between an audit the team acts on and an audit the team argues with. A finding that says 'spacing feels random on the pricing page' creates a discussion about whose taste counts. A finding that says 'pricing.tsx uses 18px and 22px gaps; the documented scale is 16px and 24px' creates a fix. The wording matters doubly because findings become engineering tickets: a ticket that says 'make chips consistent' invites subjective rework, while one that names the file, the value, and the conflicting token is actionable without further negotiation.

Code-level evidence is the easy half. For anything about rendered output — states, density, responsive wrapping, contrast — require screenshots, captured at consistent viewports, covering the states that matter: normal, hover or focus, disabled, loading, empty, error, and at least one mobile width. Many of the worst findings live at the edges: a label that wraps before its icon at 390px, a disabled state with unreadable contrast, a focus ring that vanished in one theme.

The ordering rule on the slide — observable differences first, then severity, then fix — is a guard against the model turning preferences into findings. So is the last bullet: when the system itself is ambiguous, the agent should surface the question rather than inventing a rule and auditing against it. Screenshot extraction of your own live site, diffed against documented tokens, is a useful supplementary evidence source here: every mismatch is either undocumented drift or a token that exists only in someone's head.

These pairs come straight from the school's published audit material, and the pattern is consistent: the good finding names the file or route, the value found, the value expected, and the impact, in one or two sentences. The bad finding names a feeling. Both might describe the same underlying drift; only one of them can be assigned, estimated, and closed.

The practical habit to teach here is sampling. Before anyone reads a three-hundred-finding report end to end, pull ten findings at random and check them against the evidence rule. If they hold, the report is probably trustworthy. If they read like opinions — 'the hierarchy feels weak', 'this page needs polish' — the problem is upstream in the auditor's instructions, and the fix is to tighten the subagent definition once rather than to argue with three hundred findings individually.

Note also what the good findings do not do: they do not prescribe a redesign. The finding about the duplicated cards recommends consolidation and names the variant path, but the decision about whether the system should support that variant remains a human call at the review gate. Findings prepare decisions; they do not make them.

Every audit produces false positives, and how the team handles them determines whether the second run ever happens. The most common category is intentional variation: a compact chip in a dense table genuinely needs less padding than its detail-page sibling, and an agent comparing the two will flag it unless told otherwise. The semantic state, accessible name, colour token, and icon logic should still come from the system — but the padding difference is a decision, not drift. Record approved exceptions in the audit packet so future runs stop reporting them.

The second category is style architecture. In a Tailwind-style codebase, raw utility classes are the design language, not a violation of it. A raw rounded-full on a badge may be exactly what the system intends; a raw hex colour inside a route file is far more suspicious. The auditor's instructions should distinguish token bypasses from normal composition, or the report drowns in noise.

The third is the docs-versus-code question: an agent can see that a variant exists in code and not in the documentation, but it cannot know whether the docs are behind or the variant should not exist. That distinction is a human triage call at the review gate. The general tuning loop: every false positive either becomes a recorded exception, a tightened rule in the auditor definition, or a documentation fix. What it should never become is a reason to stop running the audit.

Severity exists to stop the team spending a day tuning border radius while a warning state is unreadable on mobile. The P0-to-P3 ladder here follows the school's audit article: meaning and accessibility breaks first, behaviour and component drift second, token drift third, polish and judgment calls last. Severity is an input to planning, not a plan — roadmap pressure, ownership, and effort still shape the order — but it keeps the conversation anchored to user impact rather than to whatever finding annoyed the loudest reviewer.

The second move is grouping by fix type, because different findings route to different work. Quick fixes — replace a raw value with the token, restore a missing label — can be batched and largely automated. Component refactors, like consolidating three forked buttons, need a decision about which props survive, which is a meeting, not a commit. Token changes should be rare: add a semantic token only when the current system genuinely cannot express a real need. Documentation updates record the approved variants the audit surfaced. Human decisions cover the places where the system itself is unclear.

Ownership is the part that makes the queue real. A finding with a plausible owner gets scheduled; a finding addressed 'to the team' gets admired. Asking the auditor to suggest an owner per finding is cheap, and the review gate corrects the suggestions that are wrong.

The separation between auditing and fixing is a trust decision, not a ceremony. An agent that changes code while it audits is grading its own homework, and the team can no longer tell whether the report describes the system or the agent's edits. Keep the audit read-only; run the fixes as a second pass with write access, the same fan-out shape if the volume justifies it, and human review on every diff — the same review gate discipline Module 3 established for new components applies to fixes.

Within that constraint, the mechanical share of the queue is genuinely large. Replacing a raw hex with the token it shadows, restoring a missing aria-label, correcting a copied class — these are exactly the changes agents do reliably and humans do resentfully. The judgment-heavy findings — consolidating forks, deciding whether a variant should exist — stay with people, with the agent preparing the evidence and the options.

Scheduling is what turns this from a project into a practice. Save the working orchestration to .claude/workflows/ so the next run is a command rather than an afternoon of prompting, and pick triggers that match how your system changes: quarterly as a health check, after a rebrand to measure how much of it shipped, before a token migration to know what you are migrating from, or before releases as part of a visual QA sweep. A second run after the cleanup is also the cheapest way to prove the cleanup worked — falling severity counts are evidence, and evidence is what design system teams chronically lack when arguing for time.

This case is drawn from the school's published audit-at-scale workflow; present it as one traced run rather than a promise of what every team will see. The setup is the common one: the rebrand updated the visible surface — core tokens and the twelve most-used components — and the team needed to know how much of the long tail was still wearing the old brand. That is precisely the question hand audits never answer, because nobody re-reads sixty component folders after the launch pressure has passed.

The finding worth dwelling on is the eleven hard-coded colours hiding in disabled and hover states. Those are the states that get skipped under deadline and the states a quick visual scan never exercises — and they are the kind of finding that only shows up when the evidence requirement covers states, not just default renders. The nine components still importing legacy radius constants are the other quiet category: nothing looks broken until the constant is finally deleted and the build fails.

The ending is the operational point. The report became a three-sprint backlog because the findings carried severity, evidence, and owners; the rerun after cleanup gave the team a number — 88 P0 and P1 findings down to 6 — to put in front of stakeholders. Two adjacent cases from the same workflow are worth mentioning if there is time: a 28-page marketing site audited per page in under an hour, where the agency fixed the top twenty findings in one sprint, and three forked buttons that the report turned into one consolidation meeting instead of a quarter of intermittent debate.

Keep the exercise deliberately small: one dimension, one component family, read-only. The goal is not coverage; it is to experience the difference between an evidence-backed finding and an opinion, and to discover how much of the work is actually in writing the reference down. Most participants find that the hardest step is the second bullet — articulating what the system is supposed to be — which is itself a useful result, because ambiguity in the reference is exactly where their drift comes from.

Token drift is the recommended first dimension because the evidence is unambiguous: a raw value either matches a token or it does not. Accessibility and density audits need screenshots and state coverage, which is more setup than a first run warrants. If the participant has no codebase access, the field-note variant works: extract the rendered styles from their own live site and diff against the documented tokens — every mismatch is either undocumented drift or a token that exists only in someone's head.

The sorting step at the end is the bridge to the rest of the course. Mechanical fixes are the future automated passes; the human decisions are the review gate from Module 3 doing its job; and the rules worth adding to the harness are how this audit stops finding the same things twice. Ask participants to keep the report: Module 6 builds the scheduled, self-maintaining version of this loop, and a real first report makes that module concrete.

Recap by connecting the bullets rather than repeating them. Drift happens because the system is used faster than it is watched, so the counterweight has to be cheap enough to repeat — that is what the fan-out buys. Evidence is what makes the output trustworthy, severity and ownership are what make it actionable, and the read-only-audit-then-reviewed-fixes split is what keeps both honest. The audit also feeds the earlier modules: recurring findings become token rules from Module 1, DESIGN.md corrections from Module 2, and review-gate checks from Module 3.

Be clear about the limits one more time, because this is where overclaiming erodes trust: the audit finds drift, it does not decide what the system should be. Intentional variation, undocumented-versus-unwanted variants, and density disagreements between teams are design decisions that the report should carry as questions. Static reading also misses runtime-only issues — computed styles, theme switching, third-party widgets — which is why heavy releases pair this audit with a screenshot-based regression sweep.

Preview Module 5 concretely: most teams hold tokens in at least three places — a design tool, a canvas or spec, and code — and that multiplicity is its own drift surface. The module covers choosing a single source of truth, running sync as an agent job that reads, diffs, and proposes rather than overwrites, and reporting divergence instead of silently resolving it. The audit habits from this module carry straight across.