Frontend Beauty
ESC

Type to search the knowledge base.

Automated Axe Testing Limits

What axe-core catches reliably, the large WCAG surface it cannot, and how to combine automation with manual keyboard and SR testing.

intermediate3 min read
  • accessibility
  • automated-axe

axe-core (and wrappers like jest-axe, Storybook a11y, Lighthouse) automates checks for a subset of accessibility issues. Deque estimates automation covers only part of WCAG criteria — often cited around 30–50% depending on methodology. Treat green axe as necessary, not sufficient.

Docs: axe-core, WCAG.

What automation catches well

  • Missing form labels
  • Invalid ARIA attributes / roles
  • Empty buttons / links
  • Duplicate ids
  • Some contrast issues (computed styles)
  • Document language missing
  • Certain landmark / structure issues
const results = await axe(container);
expect(results.violations).toHaveLength(0);

What it misses (high impact)

Miss Why
Focus order / traps Needs interaction
Meaningful alt text quality “dog.jpg” passes if alt exists
Keyboard operability of custom widgets Behavior
Screen reader verbosity / context Human judgment
Color alone for meaning in charts Partial
Logical heading misuse that still “has h2s” Semantics quality
Timing, motion, cognitive load Human

A perfect score on a mouse-only modal is possible if markup is clean but JS never traps focus.

False confidence patterns

  1. Running axe only on the empty shell route.
  2. Disabling rules that fire often instead of fixing.
  3. Ignoring “needs review” incomplete results.
  4. No manual pass on primary flows.

Complete strategy

CI: jsx-a11y + axe on components/routes
  + keyboard smoke checklist on PRs for new widgets
  + SR spot checks on release for critical journeys
  + occasional expert audit

See accessibility testing in CI, screen reader testing basics.

Configuring axe wisely

await new AxeBuilder({ page })
  .withTags(['wcag2a', 'wcag2aa', 'wcag21aa'])
  .disableRules(['color-contrast']) // only with alternate contrast process
  .analyze();

Prefer fixing over disabling. If disable, document the compensating control.

Footguns

  1. Celebrating Lighthouse 100 a11y while checkout is unusable by keyboard.
  2. Snapshotting violations without owning tickets.
  3. Running different rule sets locally vs CI.

Interview out-loud answer

“Axe catches many markup issues in CI, maybe a third to half of WCAG depending on what you count. It doesn’t prove keyboard or screen reader UX. I automate axe, then manually test focus and SR on critical flows.”

Incomplete results

Axe marks some checks incomplete when color contrast can’t be determined (background images, transparency). Don’t treat “0 violations” with many incompletes as a clean bill of health — open the report and sample incompletes manually.

Extra practice

Write a minimal demo in a scratch file or the playground: one happy path, one failure path, and one boundary input. If you cannot exhibit a bug that the pattern prevents, you do not own the concept yet — re-read the primary docs linked below and tighten the example until the failure is obvious.

Notes from real codebases

Teams that succeed here keep the rules mechanical: lint where possible, CI for the rest, and a short human checklist for what automation cannot see. Document exceptions with an owner name and a removal date so “temporary” escapes do not become permanent architecture.

Further reading

Related guides