Gate accessibility in CI

The check needs a URL that answers, so where it runs depends on when the site exists. A deployed site is there all the time. The site a pull request builds exists only after the build step, on the runner. Those are the two shapes on this page, and most repositories end up running both.

Where this output came from

The command output below was copied from a real run against a served site, test/fixtures/a11y/site/ on port 4173. The workflow files are written for GitHub Actions and are not executed here. A crawl needs a browser and a running server, which the docs-as-tests job does not provide. So nothing on this page is replayed by CI the way the manni meta examples are.

Check the deployed site on a schedule

This workflow checks the live site once a week, and whenever someone runs it from the Actions tab. Nothing in the repository is read, so there is no checkout step. The site is the input.

.github/workflows/a11y.yml

name: Accessibility

on:
  schedule:
    - cron: "0 6 * * 1"
  workflow_dispatch:

jobs:
  a11y:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/setup-node@v7
        with:
          node-version: 24
      - run: npx @hawkeyexl/manni a11y check https://docs.example.com/ -f github --progress

A schedule catches what a pull request cannot see. That includes content published from a CMS, a third-party widget that changed, and a page nobody has edited in a year.

Check a pull request before it deploys

To catch a finding before it ships, build and serve the site in the pull request with manni site preview. It detects the framework, runs its build, and serves the output. Then wait for the port, and point the check at it.

.github/workflows/a11y-pr.yml

name: Accessibility (pull request)

on:
  pull_request:

jobs:
  a11y:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v7
      - uses: actions/setup-node@v7
        with:
          node-version: 24
          cache: npm
      - run: npm ci
      - name: Build and serve the site
        run: npx @hawkeyexl/manni site preview --host 127.0.0.1 --port 4173 > preview.log 2>&1 &
      - name: Wait for the server
        run: |
          tries=0
          until curl -fsS http://127.0.0.1:4173/ > /dev/null; do
            tries=$((tries + 1))
            [ "$tries" -lt 600 ] || { echo "server did not answer" >&2; cat preview.log >&2; exit 1; }
            sleep 1
          done
      - run: npx @hawkeyexl/manni a11y check http://127.0.0.1:4173/ -f github --progress

The wait step is the one people leave out. A server started with & returns immediately, and the crawl then fails on a seed that is not listening yet. That is exit 2, not a violation, so the job fails loudly rather than reporting a clean site.

The wait covers the build as well, because preview builds before it serves. That is why the loop allows ten minutes rather than one. The step’s log names the framework and the command it ran, so preview.log is the file to read when the server never answers.

A build’s sitemap usually lists the production URLs rather than 127.0.0.1. Those are on another host, so the crawl drops them and follows links instead. That reaches every page the navigation links to, which on a docs site is every page.

What each piece does

• -f github writes one annotation per rule per page, and prints nothing when the site is clean.
• --progress narrates the run on stderr, with a line as each page starts and another as it finishes. A CI log has no terminal, so without it the run is silent until the report.
• Exit 1 fails the job on a remaining violation, and exit 2 fails it on a setup problem. See exit codes.
• GitHub-hosted runners on Linux, macOS and Windows ship Chrome, so no browser step is needed. A bare container needs npx playwright install --with-deps chromium first.
• With urls and severity under a11y: in manni.config.yaml, the step is a bare manni a11y check.

The packaged Action

The manni GitHub Action runs manni check by default, and manni check does not run a11y. A step with command: a11y check runs it once the site is up. The run: step above does the same.

Annotations

-f github emits one workflow command per rule per page. GitHub renders each as an annotation on the workflow run. After the first line come up to three failing elements, each with what to change and its html, then the rule’s guidance:

::error title=a11y/button-name::Buttons must have discernible text: 1 node on http://127.0.0.1:4173/about.html.%0Abutton  → Fix any of the following: Element does not have inner text that is visible to screen readers; …%0A  <button></button>%0Aaxe-core 4.13.0 button-name: Ensure buttons have discernible text%0AGuide: https://dequeuniversity.com/rules/axe/4.13/button-name?application=playwright
::error title=a11y/color-contrast::Elements must meet minimum color contrast ratio thresholds: 1 node on http://127.0.0.1:4173/about.html.%0Ap  → Fix any of the following: Element has insufficient color contrast of 2.84 …%0A  <p style="color:#999;background:#fff">Low contrast text.</p>%0Aaxe-core 4.13.0 color-contrast: Ensure the contrast between foreground and background colors meets WCAG 2 AA minimum contrast ratio thresholds%0AGuide: https://dequeuniversity.com/rules/axe/4.13/color-contrast?application=playwright
::error title=a11y/html-has-lang::<html> element must have a lang attribute: 1 node on http://127.0.0.1:4173/about.html.%0Ahtml  → Fix any of the following: The <html> element does not have a lang attribute%0A  <html><head> <meta charset="utf-8"> …%0Aaxe-core 4.13.0 html-has-lang: Ensure every HTML document has a lang attribute%0AGuide: https://dequeuniversity.com/rules/axe/4.13/html-has-lang?application=playwright
::error title=a11y/image-alt::Images must have alternative text: 1 node on http://127.0.0.1:4173/about.html.%0Aimg  → Fix any of the following: Element does not have an alt attribute; …%0A  <img src="x.png">%0Aaxe-core 4.13.0 image-alt: Ensure <img> elements have alternative text or a role of none or presentation%0AGuide: https://dequeuniversity.com/rules/axe/4.13/image-alt?application=playwright
::error title=a11y/image-alt::Images must have alternative text: 1 node on http://127.0.0.1:4173/orphan.html.%0Aimg  → Fix any of the following: Element does not have an alt attribute; …%0A  <img src="y.png">%0Aaxe-core 4.13.0 image-alt: Ensure <img> elements have alternative text or a role of none or presentation%0AGuide: https://dequeuniversity.com/rules/axe/4.13/image-alt?application=playwright

One element that fails the same rule on several pages gets one annotation. A layout or a shared component is the usual cause, and GitHub shows only so many annotations per step. The first line names the page count and the first page:

::error title=a11y/image-alt::Images must have alternative text: 1 node on 12 pages, first http://127.0.0.1:4173/about.html. The same element fails on each page, so the fix likely belongs in a shared template.%0A…

The pretty and JSON reports still list the violation under each page.

A crawled page that fails to load gets one ::error title=a11y/load:: annotation instead, naming the URL and the reason. A clean run prints nothing at all and exits 0.

There is no file or line to anchor an annotation to, because the finding is about a URL rather than a source file. The annotation names the page, the rule and the elements, and its Guide: line links the rule’s page. Search the site’s source for the html it prints to find the template. The command level is the finding’s severity, so ::error, ::warning, and ::notice are the three you will see. axe’s four impact levels are folded onto that scale when the page is read, and the map is in the reference.

For a dashboard rather than annotations, use -f json. The JSON shape carries every finding, every failing element, and a summary object with the counts.

Exit codes

Code	Meaning
0	Every page loaded and none has a violation at or above the severity floor.
1	At least one page keeps a violation, or a crawled page failed to load.
2	The run could not happen. A bad flag value, a URL that is not http(s), no URLs and no config, or no browser. So is a seed that would not load, or that redirects to a URL that is not http(s), off its host, or to an excluded path.

Wire 1 to the gate and 2 to a loud failure. That is the same split manni meta validate uses. Keep them apart, because 2 means nothing was checked. A job that treats every non-zero code as “violations found” reports a broken runner as a red site.

A seed that will not load is exit 2, because the run never started. A page found during the crawl that will not load is recorded against that page, the crawl continues, and the page counts as failed. That is exit 1.

The floor is the gate

Every violation the floor keeps fails the run, whatever its level. A page whose worst finding is a warning exits 1, and so does one whose worst finding is a notice. --severity is the only thing that changes that.

This is the one place a11y reads the family scale differently from manni meta validate and manni cite check, which fail on error-severity findings alone. Those two let you move a finding onto the level you want, so pinning their exit code to error costs nothing. axe decides a11y’s severities from its own impact, and no config moves one. A fixed gate would leave no way to fail a build on an axe moderate finding, so --severity does both jobs here. Proposal 0064 is the record.

When the crawl runs long

Pages are checked one at a time, in a real browser, so the wall clock is roughly a second or two per page. A few hundred pages is a coffee break. Five things shorten it, in the order worth trying.

Leave out what the team does not own. --exclude "/proposals/**" keeps a section out of the crawl. It is repeatable, one glob per occurrence, and the glob matches the URL’s path, so one pattern serves a local preview and production. The footer says how many pages went, and --progress says it once before the browser starts.

Bound the run. --max-pages <n> stops once that many pages have been checked. Whatever is still queued is reported as skipped rather than failed, so the exit code still reflects what was actually looked at:

Checked 2 of 3 pages (sitemap: http://127.0.0.1:4173/sitemap.xml)
✗ http://127.0.0.1:4173/about.html  score 78  4 errors
    error  button-name (axe: critical)  1 node  Buttons must have discernible text
      axe-core 4.13.0 button-name: Ensure buttons have discernible text
      Guide: https://dequeuniversity.com/rules/axe/4.13/button-name?application=playwright
      button  → Fix any of the following: Element does not have inner text that is visible to screen readers; …
        <button></button>
    error  color-contrast (axe: serious)  1 node  Elements must meet minimum color contrast ratio thresholds
      axe-core 4.13.0 color-contrast: Ensure the contrast between foreground and background colors meets WCAG 2 AA minimum contrast ratio thresholds
      Guide: https://dequeuniversity.com/rules/axe/4.13/color-contrast?application=playwright
      p  → Fix any of the following: Element has insufficient color contrast of 2.84 (foreground color: #999999, background color: #ffffff, font size: 12.0pt (16px), font weight: normal). Expected contrast ratio of 4.5:1
        <p style="color:#999;background:#fff">Low contrast text.</p>
    error  html-has-lang (axe: serious)  1 node  <html> element must have a lang attribute
      axe-core 4.13.0 html-has-lang: Ensure every HTML document has a lang attribute
      Guide: https://dequeuniversity.com/rules/axe/4.13/html-has-lang?application=playwright
      html  → Fix any of the following: The <html> element does not have a lang attribute
        <html><head> <meta charset="utf-8"> …
    error  image-alt (axe: critical)  1 node  Images must have alternative text
      axe-core 4.13.0 image-alt: Ensure <img> elements have alternative text or a role of none or presentation
      Guide: https://dequeuniversity.com/rules/axe/4.13/image-alt?application=playwright
      img  → Fix any of the following: Element does not have an alt attribute; …
        <img src="x.png">

4 violations on 1 of 2 pages; 1 skipped (--max-pages)

The cap is on the pull request job, where the wait matters. The scheduled run has all night, so leave it uncapped there and let it see the whole site.

Narrow the seeds. A repository that declares its documents as collections can give each one a url:, then check one at a time with --collection guides. Crawl scope is still the host, so this changes where the crawl starts rather than where it may go.

Check exactly what changed. --no-crawl skips the sitemap lookup and the link following, and checks only the URLs you name. A job that maps changed files to published URLs can hand them straight to the command.

Narrow the rule set. --tags wcag2a,wcag2aa,wcag21aa restricts axe to rules carrying any of those tags, so the gate is “WCAG 2.1 AA and nothing else”. It saves less time than the other three, and it makes the gate mean something specific, which is the better reason to reach for it.

--timeout <ms> is the one that does not shorten a run. It raises the per-page navigation limit, and the default is 30 seconds. Raise it when a page is slow to render rather than when the crawl as a whole is long.

Move the settings into config

Every flag above has a config key except -f, --progress, and --collection. So the workflow step stays one line while the settings live in the repository beside meta::

manni.config.yaml

a11y:
  urls: ["https://docs.example.com/"]
  maxPages: 200
  tags: ["wcag2a", "wcag2aa", "wcag21aa"]
  severity: error
  timeout: 30000

      - run: npx @hawkeyexl/manni a11y check -f github --progress

A flag on the command line wins over the same key in config, and config wins over the default. The full table of keys, types and defaults is in the CLI reference.

Choosing the floor

severity: error is the usual first gate. It counts axe’s critical and serious findings and drops the rest before the score and the exit code are decided. Lower the floor once the errors are at zero and staying there.

It is a ratchet. Lower it to warning, and then to notice, as each level reaches zero and stays there. Raising it back to get a build green buys the same silence as deleting the job.

Get startedInstall a browser, crawl a local build, read the score and the findings, and fix one before you wire the gate.

Fix a failing checkWhat a contributor does with the annotation your job produced. One violation end to end, and the local run that proves the fix.

CLI referenceEvery argument and flag of manni a11y check, the three output formats, exit codes, and the config keys.

manni meta in CIThe metadata gate that shares the bin, the config file, and the exit-code contract this page follows.