-
Notifications
You must be signed in to change notification settings - Fork 382
Implement automated accessibility testing using Axe #3294
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
25dbf3b
4882481
be21244
08c8a41
7953557
b0e13de
3f1ec9f
e4b6b3b
052ca8a
5c7cfd5
78161f9
859eb93
8fee020
e4f953e
1eccc20
60ac04b
f08f6ee
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -122,6 +122,83 @@ function observeConsoleLogging() { | |
| } ); | ||
| } | ||
|
|
||
| /** | ||
| * Runs Axe tests when the story editor is found on the current page. | ||
| * | ||
| * @return {?Promise} Promise resolving once Axe texts are finished. | ||
| */ | ||
| async function runAxeTestsForStoriesEditor() { | ||
| if ( ! await page.$( '#amp-story-editor' ) ) { | ||
| return; | ||
| } | ||
|
|
||
| await expect( page ).toPassAxeTests( { | ||
| /** | ||
| * Rules are disabled, as there are still accessibility issues within gutenberg. | ||
| * | ||
| * See: https://github.com/WordPress/gutenberg/pull/15018 & https://github.com/WordPress/gutenberg/issues/15452 | ||
| */ | ||
| disabledRules: [ | ||
| 'aria-allowed-role', | ||
| 'aria-hidden-focus', | ||
| 'button-name', | ||
| 'color-contrast', | ||
| 'duplicate-id', | ||
| 'region', | ||
| ], | ||
| exclude: [ | ||
| // Ignores elements created by metaboxes. | ||
| '.edit-post-layout__metaboxes', | ||
| // Ignores elements created by TinyMCE. | ||
| '.mce-container', | ||
| // Ignore attachment close button. | ||
| '.amp-story-page-attachment-close-button', | ||
| ], | ||
| } ); | ||
| } | ||
|
|
||
| /** | ||
| * Runs Axe tests when the block editor is found on the current page. | ||
| * | ||
| * @return {?Promise} Promise resolving once Axe texts are finished. | ||
| */ | ||
| async function runAxeTestsForBlockEditor() { | ||
| if ( ! await page.$( '.block-editor' ) ) { | ||
|
spacedmonkey marked this conversation as resolved.
|
||
| return; | ||
| } | ||
|
|
||
| // If amp story editor is found, don't run tests. | ||
| if ( await page.$( '#amp-story-editor' ) ) { | ||
| return; | ||
| } | ||
|
|
||
| await expect( page ).toPassAxeTests( { | ||
| /** | ||
| * Rules are disabled, as there are still accessibility issues within gutenberg. | ||
| * | ||
| * See: https://github.com/WordPress/gutenberg/pull/15018 & https://github.com/WordPress/gutenberg/issues/15452 | ||
| */ | ||
| disabledRules: [ | ||
| 'aria-allowed-role', | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. These rules are disabled because of the errors in gutenberg. I would love for this ignored rules to be removed. |
||
| 'aria-valid-attr-value', | ||
| 'button-name', | ||
| 'color-contrast', | ||
| 'dlitem', | ||
| 'duplicate-id', | ||
| 'label', | ||
| 'link-name', | ||
| 'listitem', | ||
| 'region', | ||
| ], | ||
| exclude: [ | ||
| // Ignores elements created by metaboxes. | ||
| '.edit-post-layout__metaboxes', | ||
| // Ignores elements created by TinyMCE. | ||
| '.mce-container', | ||
| ], | ||
| } ); | ||
| } | ||
|
|
||
| /** | ||
| * Before every test suite run, delete all content created by the test. This ensures | ||
| * other posts/comments/etc. aren't dirtying tests and tests don't depend on | ||
|
|
@@ -140,6 +217,8 @@ beforeAll( async () => { | |
| // eslint-disable-next-line jest/require-top-level-describe | ||
| afterEach( async () => { | ||
| await clearLocalStorage(); | ||
| await runAxeTestsForStoriesEditor(); | ||
| await runAxeTestsForBlockEditor(); | ||
| await setBrowserViewport( 'large' ); | ||
| } ); | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -104,7 +104,7 @@ describe( 'Block Position Controls', () => { | |
|
|
||
| // Since the selected block is already at the back, the 'Back' and 'Backward' buttons should be disabled. | ||
| expect( page ).toMatchElement( '.amp-story-controls-send-back[aria-disabled="true"]' ); | ||
| expect( page ).toMatchElement( '.amp-story-controls-send-backward[aria-disabled="true"]' ); | ||
| expect( page ).toMatchElement( '.amp-story-controls-send-backwards[aria-disabled="true"]' ); | ||
|
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This is a fix for e2e to pass. Not sure how this was passing before. |
||
| } ); | ||
| } ); | ||
| } ); | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we rename this to
runAxeTestsForStoriesEditorand use a different selector to detect it?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@swissspidy Why? The tests are passing for both. Any new e2e added for plugin should also run these axe tests.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The block editor and stories editor will have different configurations though. The
excludeconfig already has a stories-specific config in it as well.Also, eventually we should eventually add these tests for the plugin settings screen as well, where we're going to have yet another set of configuration options.
Having three separate sets would make maintenance easier.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fixed in 78161f9
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The selector in
runAxeTestsForStoriesEditoris still the same though.What I meant was having separate functions with separate configurations:
runAxeTestsForBlockEditorwith the current.block-editorselectorrunAxeTestsForStoriesEditorwith a#amp-story-editorselectorrunAxeTestsForSettingsScreenwith a#amp-settingsselectorrunAxeTestsForValidationScreenwith a.post-type-amp_validated_urlselectorThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What are you asking to change here? Just change the selector, like this
runAxeTestsForStoriesEditorwith a#amp-story-editorselectorThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. And have a second function,
runAxeTestsForBlockEditor, with a different selector that only targets the block editor but not the stories editor. That function would then notexclude'.amp-story-page-attachment-close-button'as it's irrelevant in the standalone block editor.If it's unclear I can also change this myself.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And of course, let me know if you think it doesn't make sense. Then we can just do the
#amp-story-editorselector change for now.