Automated WCAG 2.1 AA a11y testing across all themes/modes by sjoerdbeentjes · Pull Request #38 · SURFnet/DesignSystem

sjoerdbeentjes · 2026-06-22T13:06:08Z

Adds an automated WCAG 2.1 AA pipeline that sweeps every Storybook story across all themes and modes from @surfnet/tokens, in both React and Angular.

What

Shared config in @surfnet/storybook-config: browser-safe a11yParameters (merged into both previews, also scopes the addon-a11y panel) and a node-only runStoryA11yAudit behind a ./test-runner subpath so it never enters a preview bundle.
Builder-agnostic runner: @storybook/test-runner + axe-playwright against a served Storybook, so one setup covers React (Vite) and Angular (webpack). Each story is audited once per theme × mode (11 themes × 2 modes), writing a per-story JSON report.
Wiring: per-package test:a11y / test:a11y:ci scripts, a turbo task, a root script, and a CI step that runs report-only (continue-on-error) and uploads the JSON as an artifact.

Notes

CI is intentionally report-only until the token contrast backlog is triaged; flip to blocking by removing continue-on-error.

github-actions · 2026-06-22T14:19:16Z

♿ Accessibility audit — WCAG 2.1 AA

⚠️ 41 finding(s) — 41 contrast (per theme·mode), 0 theme-independent · across 7 stories / 154 audited combinations. Report-only — does not block merge.

@surfnet/react — 41 finding(s)

Button — 41 finding(s)

As Link — 10 finding(s)

default
- dark — #04389a on #0a0a0a · ratio 1.92 — color-contrast (1.4.3)
groenvermogen-nkph2
- light — #84cc16 on #ffffff · ratio 1.97 — color-contrast (1.4.3)
studielink-aii
- dark — #2a3795 on #0a0a0a · ratio 1.95 — color-contrast (1.4.3)
surf-blue-purple
- dark — #04389a on #0a0a0a · ratio 1.92 — color-contrast (1.4.3)
surf-blue-turquoise
- dark — #04389a on #0a0a0a · ratio 1.92 — color-contrast (1.4.3)
surf-green
- light — #00d37f on #ffffff · ratio 1.97 — color-contrast (1.4.3)
surf-orange
- light — #ff883f on #ffffff · ratio 2.37 — color-contrast (1.4.3)
surf-purple
- dark — #420984 on #0a0a0a · ratio 1.51 — color-contrast (1.4.3)
surf-purple-orange
- dark — #420984 on #0a0a0a · ratio 1.51 — color-contrast (1.4.3)
surf-yellow
- light — #ffe500 on #ffffff · ratio 1.27 — color-contrast (1.4.3)

Default — 1 finding(s)

groenvermogen-nkph2
- light — #fafafa on #84cc16 · ratio 1.89 — color-contrast (1.4.3)

Sizes — 1 finding(s)

groenvermogen-nkph2
- light — #fafafa on #84cc16 · ratio 1.89 — color-contrast (1.4.3)

Variants — 20 finding(s)

default
- light — #000000 on #2e6ee5 · ratio 4.47 — color-contrast (1.4.3)
- dark — #04389a on #0a0a0a · ratio 1.92 — color-contrast (1.4.3)
groenvermogen-nkph2
- light — #fafafa on #84cc16 · ratio 1.89 — color-contrast (1.4.3)
shadcn-default
- light — #dc2626 on #fce9e9 · ratio 4.13 — color-contrast (1.4.3)
studielink-aii
- light — #dc2626 on #fce9e9 · ratio 4.13 — color-contrast (1.4.3)
- dark — #2a3795 on #0a0a0a · ratio 1.95 — color-contrast (1.4.3)
surf-blue-purple
- light — #dc2626 on #fce9e9 · ratio 4.13 — color-contrast (1.4.3)
- dark — #ffffff on #9d5de7 · ratio 4.07 — color-contrast (1.4.3)
surf-blue-turquoise
- light — #dc2626 on #fce9e9 · ratio 4.13 — color-contrast (1.4.3)
- dark — #ffffff on #39bcb1 · ratio 2.33 — color-contrast (1.4.3)
surf-green
- light — #dc2626 on #fce9e9 · ratio 4.13 — color-contrast (1.4.3)
- dark — #ffffff on #31e19b · ratio 1.69 — color-contrast (1.4.3)
surf-orange
- light — #dc2626 on #fce9e9 · ratio 4.13 — color-contrast (1.4.3)
- dark — #ffffff on #ff883f · ratio 2.37 — color-contrast (1.4.3)
surf-purple
- light — #dc2626 on #fce9e9 · ratio 4.13 — color-contrast (1.4.3)
- dark — #ffffff on #9d5de7 · ratio 4.07 — color-contrast (1.4.3)
surf-purple-orange
- light — #dc2626 on #fce9e9 · ratio 4.13 — color-contrast (1.4.3)
- dark — #ffffff on #ff883f · ratio 2.37 — color-contrast (1.4.3)
surf-yellow
- light — #dc2626 on #fce9e9 · ratio 4.13 — color-contrast (1.4.3)
- dark — #ffffff on #ffe84a · ratio 1.24 — color-contrast (1.4.3)

With Icon — 9 finding(s)

default
- light — #000000 on #2e6ee5 · ratio 4.47 — color-contrast (1.4.3)
groenvermogen-nkph2
- light — #fafafa on #84cc16 · ratio 1.89 — color-contrast (1.4.3)
surf-blue-purple
- dark — #ffffff on #9d5de7 · ratio 4.07 — color-contrast (1.4.3)
surf-blue-turquoise
- dark — #ffffff on #39bcb1 · ratio 2.33 — color-contrast (1.4.3)
surf-green
- dark — #ffffff on #31e19b · ratio 1.69 — color-contrast (1.4.3)
surf-orange
- dark — #ffffff on #ff883f · ratio 2.37 — color-contrast (1.4.3)
surf-purple
- dark — #ffffff on #9d5de7 · ratio 4.07 — color-contrast (1.4.3)
surf-purple-orange
- dark — #ffffff on #ff883f · ratio 2.37 — color-contrast (1.4.3)
surf-yellow
- dark — #ffffff on #ffe84a · ratio 1.24 — color-contrast (1.4.3)

Automated axe covers ~30–50% of WCAG 2.1 AA. Keyboard, screen-reader and reflow checks still need a manual pass. Full per-story JSON is in the run’s a11y-reports artifact.

sjoerdbeentjes · 2026-06-23T07:23:26Z

Example report from earlier test run with forced violations:

♿ Accessibility audit — WCAG 2.1 AA

⚠️ 87 finding(s) — 82 contrast (per theme·mode), 5 theme-independent · across 10 stories / 220 audited combinations. Report-only — does not block merge.

@surfnet/react — 87 finding(s)

Button — 87 finding(s)

A11Y Violation Check — 3 finding(s)

Buttons must have discernible text — button-name (4.1.2) · all themes/modes
Images must have alternative text — image-alt (1.1.1) · all themes/modes
Links must have discernible text — link-name (4.1.2) · all themes/modes

As Link — 10 finding(s)

default
- dark — #04389a on #0a0a0a · ratio 1.92 — color-contrast (1.4.3)
groenvermogen-nkph2
- light — #84cc16 on #ffffff · ratio 1.97 — color-contrast (1.4.3)
studielink-aii
- dark — #2a3795 on #0a0a0a · ratio 1.95 — color-contrast (1.4.3)
surf-blue-purple
- dark — #04389a on #0a0a0a · ratio 1.92 — color-contrast (1.4.3)
surf-blue-turquoise
- dark — #04389a on #0a0a0a · ratio 1.92 — color-contrast (1.4.3)
surf-green
- light — #00d37f on #ffffff · ratio 1.97 — color-contrast (1.4.3)
surf-orange
- light — #ff883f on #ffffff · ratio 2.37 — color-contrast (1.4.3)
surf-purple
- dark — #420984 on #0a0a0a · ratio 1.51 — color-contrast (1.4.3)
surf-purple-orange
- dark — #420984 on #0a0a0a · ratio 1.51 — color-contrast (1.4.3)
surf-yellow
- light — #ffe500 on #ffffff · ratio 1.27 — color-contrast (1.4.3)

Default — 1 finding(s)

groenvermogen-nkph2
- light — #fafafa on #84cc16 · ratio 1.89 — color-contrast (1.4.3)

Link In Text Block Check — 22 finding(s)

default
- light — Links must be distinguishable without relying on color — link-in-text-block (1.4.1)
- dark — Links must be distinguishable without relying on color — link-in-text-block (1.4.1)
groenvermogen-nkph2
- light — Links must be distinguishable without relying on color — link-in-text-block (1.4.1)
- dark — Links must be distinguishable without relying on color — link-in-text-block (1.4.1)
shadcn-default
- light — Links must be distinguishable without relying on color — link-in-text-block (1.4.1)
- dark — Links must be distinguishable without relying on color — link-in-text-block (1.4.1)
studielink-aii
- light — Links must be distinguishable without relying on color — link-in-text-block (1.4.1)
- dark — Links must be distinguishable without relying on color — link-in-text-block (1.4.1)
surf-blue-purple
- light — Links must be distinguishable without relying on color — link-in-text-block (1.4.1)
- dark — Links must be distinguishable without relying on color — link-in-text-block (1.4.1)
surf-blue-turquoise
- light — Links must be distinguishable without relying on color — link-in-text-block (1.4.1)
- dark — Links must be distinguishable without relying on color — link-in-text-block (1.4.1)
surf-green
- light — Links must be distinguishable without relying on color — link-in-text-block (1.4.1)
- dark — Links must be distinguishable without relying on color — link-in-text-block (1.4.1)
surf-orange
- light — Links must be distinguishable without relying on color — link-in-text-block (1.4.1)
- dark — Links must be distinguishable without relying on color — link-in-text-block (1.4.1)
surf-purple
- light — Links must be distinguishable without relying on color — link-in-text-block (1.4.1)
- dark — Links must be distinguishable without relying on color — link-in-text-block (1.4.1)
surf-purple-orange
- light — Links must be distinguishable without relying on color — link-in-text-block (1.4.1)
- dark — Links must be distinguishable without relying on color — link-in-text-block (1.4.1)
surf-yellow
- light — Links must be distinguishable without relying on color — link-in-text-block (1.4.1)
- dark — Links must be distinguishable without relying on color — link-in-text-block (1.4.1)

Mixed Violations Check — 21 finding(s)

Buttons must have discernible text — button-name (4.1.2) · all themes/modes
Images must have alternative text — image-alt (1.1.1) · all themes/modes
default
- light — #d6d3d1 on #ffffff · ratio 1.48 — color-contrast (1.4.3)
- dark — #232323 on #0a0a0a · ratio 1.25 — color-contrast (1.4.3)
groenvermogen-nkph2
- light — #e5e5e5 on #ffffff · ratio 1.25 — color-contrast (1.4.3)
shadcn-default
- light — #e5e5e5 on #ffffff · ratio 1.25 — color-contrast (1.4.3)
studielink-aii
- light — #e5e5e5 on #ffffff · ratio 1.25 — color-contrast (1.4.3)
surf-blue-purple
- light — #d6d3d1 on #ffffff · ratio 1.48 — color-contrast (1.4.3)
- dark — #232323 on #0a0a0a · ratio 1.25 — color-contrast (1.4.3)
surf-blue-turquoise
- light — #d6d3d1 on #ffffff · ratio 1.48 — color-contrast (1.4.3)
- dark — #232323 on #0a0a0a · ratio 1.25 — color-contrast (1.4.3)
surf-green
- light — #d6d3d1 on #ffffff · ratio 1.48 — color-contrast (1.4.3)
- dark — #232323 on #0a0a0a · ratio 1.25 — color-contrast (1.4.3)
surf-orange
- light — #d6d3d1 on #ffffff · ratio 1.48 — color-contrast (1.4.3)
- dark — #232323 on #0a0a0a · ratio 1.25 — color-contrast (1.4.3)
surf-purple
- light — #d6d3d1 on #ffffff · ratio 1.48 — color-contrast (1.4.3)
- dark — #232323 on #0a0a0a · ratio 1.25 — color-contrast (1.4.3)
surf-purple-orange
- light — #d6d3d1 on #ffffff · ratio 1.48 — color-contrast (1.4.3)
- dark — #232323 on #0a0a0a · ratio 1.25 — color-contrast (1.4.3)
surf-yellow
- light — #d6d3d1 on #ffffff · ratio 1.48 — color-contrast (1.4.3)
- dark — #232323 on #0a0a0a · ratio 1.25 — color-contrast (1.4.3)

Sizes — 1 finding(s)

groenvermogen-nkph2
- light — #fafafa on #84cc16 · ratio 1.89 — color-contrast (1.4.3)

Variants — 20 finding(s)

default
- light — #000000 on #2e6ee5 · ratio 4.47 — color-contrast (1.4.3)
- dark — #04389a on #0a0a0a · ratio 1.92 — color-contrast (1.4.3)
groenvermogen-nkph2
- light — #fafafa on #84cc16 · ratio 1.89 — color-contrast (1.4.3)
shadcn-default
- light — #dc2626 on #fce9e9 · ratio 4.13 — color-contrast (1.4.3)
studielink-aii
- light — #dc2626 on #fce9e9 · ratio 4.13 — color-contrast (1.4.3)
- dark — #2a3795 on #0a0a0a · ratio 1.95 — color-contrast (1.4.3)
surf-blue-purple
- light — #dc2626 on #fce9e9 · ratio 4.13 — color-contrast (1.4.3)
- dark — #ffffff on #9d5de7 · ratio 4.07 — color-contrast (1.4.3)
surf-blue-turquoise
- light — #dc2626 on #fce9e9 · ratio 4.13 — color-contrast (1.4.3)
- dark — #ffffff on #39bcb1 · ratio 2.33 — color-contrast (1.4.3)
surf-green
- light — #dc2626 on #fce9e9 · ratio 4.13 — color-contrast (1.4.3)
- dark — #ffffff on #31e19b · ratio 1.69 — color-contrast (1.4.3)
surf-orange
- light — #dc2626 on #fce9e9 · ratio 4.13 — color-contrast (1.4.3)
- dark — #ffffff on #ff883f · ratio 2.37 — color-contrast (1.4.3)
surf-purple
- light — #dc2626 on #fce9e9 · ratio 4.13 — color-contrast (1.4.3)
- dark — #ffffff on #9d5de7 · ratio 4.07 — color-contrast (1.4.3)
surf-purple-orange
- light — #dc2626 on #fce9e9 · ratio 4.13 — color-contrast (1.4.3)
- dark — #ffffff on #ff883f · ratio 2.37 — color-contrast (1.4.3)
surf-yellow
- light — #dc2626 on #fce9e9 · ratio 4.13 — color-contrast (1.4.3)
- dark — #ffffff on #ffe84a · ratio 1.24 — color-contrast (1.4.3)

With Icon — 9 finding(s)

default
- light — #000000 on #2e6ee5 · ratio 4.47 — color-contrast (1.4.3)
groenvermogen-nkph2
- light — #fafafa on #84cc16 · ratio 1.89 — color-contrast (1.4.3)
surf-blue-purple
- dark — #ffffff on #9d5de7 · ratio 4.07 — color-contrast (1.4.3)
surf-blue-turquoise
- dark — #ffffff on #39bcb1 · ratio 2.33 — color-contrast (1.4.3)
surf-green
- dark — #ffffff on #31e19b · ratio 1.69 — color-contrast (1.4.3)
surf-orange
- dark — #ffffff on #ff883f · ratio 2.37 — color-contrast (1.4.3)
surf-purple
- dark — #ffffff on #9d5de7 · ratio 4.07 — color-contrast (1.4.3)
surf-purple-orange
- dark — #ffffff on #ff883f · ratio 2.37 — color-contrast (1.4.3)
surf-yellow
- dark — #ffffff on #ffe84a · ratio 1.24 — color-contrast (1.4.3)

Automated axe covers ~30–50% of WCAG 2.1 AA. Keyboard, screen-reader and reflow checks still need a manual pass. Full per-story JSON is in the run’s a11y-reports artifact.

Dot-prefix the per-package report output (and the generated PR-comment markdown) so a single .gitignore entry covers them and they read as generated/hidden. Updates the audit default dir, the comment script's read/write paths, turbo outputs, the CI artifact/comment paths, and .gitignore.

Replace the flat findings table with a nested layout (component heading, theme bullet, mode sub-bullet with the contrast detail) so reviewers can scan by component and drill into a theme. Cap body length under GitHub's comment size limit.

Show the component (e.g. Button) as an h4 and each story variation (e.g. Variants, As Link) as an h5 beneath it, instead of merging them into one "Button / Variants" heading. Theme/mode nesting is unchanged.

Only contrast rules (color-contrast, link-in-text-block) depend on the theme/mode, so report those per theme·mode as before. DOM-structural rules (missing alt, button/link name, ARIA, etc.) are identical across themes — collapse each to a single finding per story variation, flagged 'all themes/modes', instead of repeating it for every combination.

…accordingly

Disable CSS transitions/animations and await fonts before the axe sweep so contrast reads settled colors and correct large-text thresholds. Condense the verbose comments added across the a11y tooling.

sjoerdbeentjes force-pushed the feat/a11y-testing branch 5 times, most recently from b563368 to 871c744 Compare June 22, 2026 14:18

sjoerdbeentjes force-pushed the feat/a11y-testing branch from 65e29e3 to 70b2499 Compare June 23, 2026 07:28

sjoerdbeentjes linked an issue Jun 23, 2026 that may be closed by this pull request

WCAG 2.1 AA-check op alle PoC-componenten #22

Open

sjoerdbeentjes force-pushed the feat/a11y-testing branch from 70b2499 to 4e73d0f Compare June 23, 2026 13:14

sjoerdbeentjes added 11 commits June 24, 2026 13:02

Add a11y config and audit module to @surfnet/storybook-config

673292d

Wire WCAG 2.1 AA audit into React and Angular Storybooks

97be22a

Add test:a11y workspace script, Turbo task, and CI step

d1026e3

Reorder storybook:serve before test:a11y in package scripts

b68306f

Split component and variation into h4/h5 in a11y comment

8eed160

Show the component (e.g. Button) as an h4 and each story variation (e.g. Variants, As Link) as an h5 beneath it, instead of merging them into one "Button / Variants" heading. Theme/mode nesting is unchanged.

Add link-in-text-block WCAG ref (1.4.1) to a11y comment

24850d8

Move playwright dep from react to storybook-config, scope CI install …

c935f93

…accordingly

Stabilise a11y audit and trim comments

8b13cff

Disable CSS transitions/animations and await fonts before the axe sweep so contrast reads settled colors and correct large-text thresholds. Condense the verbose comments added across the a11y tooling.

sjoerdbeentjes force-pushed the feat/a11y-testing branch from 4e73d0f to 8b13cff Compare June 24, 2026 11:07

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Automated WCAG 2.1 AA a11y testing across all themes/modes#38

Automated WCAG 2.1 AA a11y testing across all themes/modes#38
sjoerdbeentjes wants to merge 11 commits into
mainfrom
feat/a11y-testing

sjoerdbeentjes commented Jun 22, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Jun 22, 2026 •

edited

Loading

Button — 41 finding(s)

As Link — 10 finding(s)

Default — 1 finding(s)

Sizes — 1 finding(s)

Variants — 20 finding(s)

With Icon — 9 finding(s)

Uh oh!

sjoerdbeentjes commented Jun 23, 2026 •

edited

Loading

Button — 87 finding(s)

A11Y Violation Check — 3 finding(s)

As Link — 10 finding(s)

Default — 1 finding(s)

Link In Text Block Check — 22 finding(s)

Mixed Violations Check — 21 finding(s)

Sizes — 1 finding(s)

Variants — 20 finding(s)

With Icon — 9 finding(s)

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

sjoerdbeentjes commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Notes

Uh oh!

github-actions Bot commented Jun 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

♿ Accessibility audit — WCAG 2.1 AA

Button — 41 finding(s)

As Link — 10 finding(s)

Default — 1 finding(s)

Sizes — 1 finding(s)

Variants — 20 finding(s)

With Icon — 9 finding(s)

Uh oh!

sjoerdbeentjes commented Jun 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

♿ Accessibility audit — WCAG 2.1 AA

Button — 87 finding(s)

A11Y Violation Check — 3 finding(s)

As Link — 10 finding(s)

Default — 1 finding(s)

Link In Text Block Check — 22 finding(s)

Mixed Violations Check — 21 finding(s)

Sizes — 1 finding(s)

Variants — 20 finding(s)

With Icon — 9 finding(s)

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

sjoerdbeentjes commented Jun 22, 2026 •

edited

Loading

github-actions Bot commented Jun 22, 2026 •

edited

Loading

sjoerdbeentjes commented Jun 23, 2026 •

edited

Loading