UI Acceptance Testing Accessibility

Accessibility keywords

In the previous post Unit Testing Accessibility I showed how to run accessibility check on HTML node with aXe. This approach can be used to test components of your website.

You can take your accessibility testing to the next level by adding accessibility check for entire pages.

Do you remember Martin Fowler’s testing pyramid?

testign pyramid

While high-level UI tests can detect the same issues that unit tests can, usually UI tests are slower to run. Sometimes it takes 30 seconds to 1 minute to run 1 UI test, while unit test can be run in less than 100 milliseconds. Ideally you should have the most common scenario covered by UI test, and all possible customizations (on the component level) covered by unit tests. You can also add UI tests for some complex combinations of your components, and when you are fixing a bug in situation that unit test cannot cover encountered scenario. Because as you know, while fixing a bug, you should add unit test covering buggy scenario.

In general, unit tests and UI tests should be complementary. UI test should indicate an issue, while unit test should help you to find a source of the problem.

In accessibility testing, high-level checks are very useful in detecting accessibility issues caused by some “small change”. Once you detect accessibility violation, you should:

  1. narrow it down to particular unit of your website
  2. add unit test to cover broken scenario
  3. make sure it fails
  4. fix the issue (by writing code)
  5. make sure that unit test pass
  6. make sure that end to end UI tests pass

Check out Marcy Sutton’s article: Accessibility Testing with aXe and WebdriverJS. She created sample github repo that is demonstrating how to set everything up.

Unit Testing Accessibility

In Web Accessibility Hacker Way I mentioned that “only 20% of accessibility requirements can be verified by tools”. Nevertheless, it is worth to cover this 20%. Especially, when it is not very hard. You know that having automated test that guard against regressions always pays off in a long run.

As of today the best automatic verification tool for accessibility is aXe.


There is aXe Chrome plugin and aXe Firefox plugin that enables you to run accessibility audit manually:

aXe - results

Running automated tool manually is useful, but it is better to run it automatically as unit test, and incorporate it into your Continuous Integration to run it automatically after every commit.

Running accessibility audit with aXe

You can install aXe with npm:

npm i axe-core

The aXe has a function a11yCheck that performs accessibility audit on specified HTML Element. You may run it against widgets or partial views on your web app. That function takes 2 parameters:

  1. HTMLElement to be audited
  2. callback function that is invoked with results parameter
axe.a11yCheck($("#myElement")[0], function (results) {
    results.violations.length && console.error(results.violations);

It is useful to print errors to console, as the results.violations is an array of nested objects with different properties. Many of them are helpful to diagnose the issue.

aXe - console errors

*a11y is abbreviation for accessibility (similar like i18n for internationalization), 11 is number of letters between ‘a’ and ‘y’

Running aXe with Jasmine 2.x

In order to run aXe with Jasmine, you need to take into account that a11yCheck is asynchronous. Thus, you need to pass and invoke done function:

describe("a11y check", function() {
  it("has no accessbility violations (check console for errors)", function(done) {
    axe.a11yCheck($("#myElement")[0], function (results) {
        if (results.violations.length>0) {

Running aXe with QUnit 2.x

It is similar in QUnit. You also need to invoke done function, but first you need to get it by calling assert.async():

QUnit.test("a11y check", function(assert) {
    var done = assert.async();

    axe.a11yCheck($("#myElement")[0], function (results) {
        assert.strictEqual(results.violations.length, 0, "There should be no A11y violations (check console for errors)");
        if (results.violations.length>0) {


I created a sample with button and input tag:

<div id="fixture">
  <button>My button</button> 
  <input type="text" />

This sample is not accessible because input tag does not have a label, and a11yCheck should report violations. The sample code, with Jasmine and QUnit tests, is available on github: axe-unittests.


While automated accessibility unit tests are great, you probably still want to use aXe plugin for Chrome to investigate reported violations. It’s more convenient, and it has neat user interface, while in unit tests you need to dig in into console errors.

When you start adding accessibility checks for different parts of your system, you may encounter many violations at first. It is better to still add tests, that ignore known violations, and then incrementally fix the issues. This approach prevents regressions, while delaying adding test until all violations are fixed may cause introducing new ones while fixing others.

Web Accessibility Hacker Way

web accessibility meme

Have you heard about accessibility? Do you know what that is? Do you know what it takes to make your website accessible?

Making your website accessible means providing the ability for everyone – regardless of disability or special needs – to use it.

Unfortunately, if you put together all accessibility specifications and print it, the stack will be higher than CN Tower in Toronto:

CN Tower (Toronto)

This is not very encouraging. Many people think that it is not worth the effort because the customer base with special needs is significantly low. However, there is a thing called situational disability that applies to all of us. Moreover, when we are building our websites with accessibility in mind, we make them better for everyone. Why? Because everybody is more comfortable when allowed to fill out forms on a website using just a keyboard (without mouse) and able to switch between fields fast. Generally, everybody likes it when the focus is set on the search bar for sites because 99% of the time, the first thing you do is search for something. Everybody likes properly matched colors (accessible contrast).

How do you get started? How to survive without reading tons of specifications?

3 steps to fulfill 80% standards with 20% effort

  1. Make your website usable with keyboard only:
    • make sure that focus outline is visible all the time and user can determine which element is currently focused
    • no extra/unnecessary TAB stops
    • no tabstop traps (when you cannot get out of an element with the keyboard)
  2. Implement smart focus management:
    • set focus on appropriate elements after user actions (e.g., when a user navigates to a page with a login form – set the focus on the login text field; in 90% of the cases the next user action will be entering the login)
    • restore focus to appropriate elements after user actions (e.g., when a user closes a menu, focus should be restored to the element that was focused on before opening the menu)
    • make tab order user-friendly (remove non-actionable and non-informative tab stops)
  3. Make your website usable with screen reader:
    • When element gets focused, screen reader should provide meaningful information to the user (e.g., “image Satya Nadella” when focused on Satya’s image, or “menu collapsed” when focused on dropdown menu)
    • Min bar: make it good enough with one screen reader and one browser first. As of 2016-08-22 the best browser + screen reader combination is Firefox + NVDA (it has a text mode that prints output instead of converting it into speech: Tools -> Speech viewer)
    • Add aria tags to elements only when the screen reader cannot infer information from HTML (e.g., button element does not need role=’button’ as screen reader will infer it)

Good to know

  1. Don’t focus too much on “being compliant to the standards” at the beginning. Just use the common sense and your intuition. You can deep dive into the standards later on.
  2. Make your website’s most common scenarios work well first and focus on less popular pages later.
  3. Only 20% of accessibility requirements can be verified by tools. The rest has to be verified manually by:
    • yourself
    • user study
    • an accessibility expert

Resources to get started


If you are web developer you probably like when the tool you are using enables you to do everything without the mouse. This is thanks to keyboard accessibility, and smart focus management. If you play with color contrast analyzer you will notice that colors with good contrast are easier to read and simply just look better. Be aware that accessibility is about performance first. When your website is slow it can lead to unexpected focus behaviors, unwanted user actions, or moving content. How many times you tried to scroll to some part of website and it was moving because images were loading? I’m sure a lot.

What do you think? Is the website you are working on accessible? Have you ever even thought about it?

Accessibility all the things