a11y-agent vs axe-core

Speed and accordance benchmark comparing a11y-agent against axe-core.

Methodology

Synthetic DOMs of 100, 500, and 2k elements are generated with a realistic mix of images, forms, ARIA attributes, links, buttons, tables, lists, keyboard patterns, and media — some with intentional violations. After a warmup pass, each tool is timed over multiple iterations (auto-calibrated from 5–50 based on first-run duration). axe-core's color-contrast rule is disabled for a fair comparison since a11y-agent does not include it. Accordance is measured at the rule level on the 500-element DOM: concordance = % of a11y-agent findings confirmed by axe-core, coverage = % of axe-core findings also found by a11y-agent. Benchmarks are pre-computed at build time using happy-dom.

Concordance (500 elements)

Rule ID a11y-agent axe-core Status

Speed Comparison

DOM Size a11y-agent axe-core Speedup Relative Time

Per-Rule Breakdown (500 elements)

Rule ID Mean (ms) Violations Time
Why is a11y-agent faster?
  • No virtual DOM. axe-core builds a full VirtualNode tree of the entire document before any rule runs — wrapping every element with metadata like shadowId, children, parent, and a _cache object. This is O(n) over the whole DOM. a11y-agent skips this entirely and queries the real DOM directly: 
    // a11y-agent: queries only <img> elements
for (const img of doc.querySelectorAll("img")) { ... }

// axe-core: wraps entire DOM first, then queries
_getFlattenedTree(node) → VirtualNode for every element
    Tradeoff: axe-core's virtual tree exists to cross shadow DOM boundaries — native querySelectorAll can't reach into shadow roots. a11y-agent handles shadow DOM separately via createChunkedShadowAudit, which audits each shadow root individually rather than flattening the whole tree upfront.
  • Flat execution path. axe-core routes each check through 7 layers: axe.run → runRules → audit.run → rule.run → rule.runChecks → check.run → evaluate, with promise queues and option merging at each step. a11y-agent rules are single functions called in a loop: 
    // a11y-agent rule runner (src/rules/index.ts)
for (const rule of rules) {
  violations.push(...rule.run(doc));
}
    Tradeoff: axe-core's layered architecture enables pluggable checks, per-rule option overrides, and async evaluation — useful for an extensible ecosystem where third parties author rules. a11y-agent's flat loop is faster but rules must be compiled in, not loaded at runtime.
  • Lazy result serialization. axe-core eagerly computes CSS selector paths, ancestry chains, and XPath for every violation node. a11y-agent only calls getSelector() and getHtmlSnippet() at the moment a violation is found, and does no post-processing pass.
    Tradeoff: axe-core's richer metadata (ancestry, XPath) supports integrations that need precise element identification across frames and dynamic DOMs. a11y-agent produces lighter results suited for its own AI-assisted remediation workflow.
  • Narrow selectors with early exits. Each rule uses the most specific selector possible ([role], a[href], button) and exits on the first disqualifying condition (isAriaHidden, role="presentation") before any expensive computation like accessible name resolution.
    Tradeoff: axe-core's broader element gathering and multi-check architecture allows a single rule to evaluate multiple conditions (any/all/none) and report partial passes — useful for detailed compliance reporting. a11y-agent reports pass/fail per rule without granular check-level detail.
  • No frame detection overhead. axe-core scans for all <iframe> elements during context setup and recursively creates audit contexts for each, even when frame auditing isn't needed. a11y-agent operates on the document it's given.
    Tradeoff: axe-core provides seamless cross-frame auditing out of the box — violations inside iframes are automatically included in results. a11y-agent requires the caller to audit each frame separately.