unit tests

Unit Testing Accessibility

In Web Accessibility Hacker Way I mentioned that “only 20% of accessibility requirements can be verified by tools”. Nevertheless, it is worth to cover this 20%. Especially, when it is not very hard. You know that having automated test that guard against regressions always pays off in a long run.

As of today the best automatic verification tool for accessibility is aXe.

aXe

There is aXe Chrome plugin and aXe Firefox plugin that enables you to run accessibility audit manually:

aXe - results

Running automated tool manually is useful, but it is better to run it automatically as unit test, and incorporate it into your Continuous Integration to run it automatically after every commit.

Running accessibility audit with aXe

You can install aXe with npm:

npm i axe-core

The aXe has a function a11yCheck that performs accessibility audit on specified HTML Element. You may run it against widgets or partial views on your web app. That function takes 2 parameters:

  1. HTMLElement to be audited
  2. callback function that is invoked with results parameter
axe.a11yCheck($("#myElement")[0], function (results) {
    expect(results.violations.length).toBe(0);
    results.violations.length && console.error(results.violations);
});

It is useful to print errors to console, as the results.violations is an array of nested objects with different properties. Many of them are helpful to diagnose the issue.

aXe - console errors

*a11y is abbreviation for accessibility (similar like i18n for internationalization), 11 is number of letters between ‘a’ and ‘y’

Running aXe with Jasmine 2.x

In order to run aXe with Jasmine, you need to take into account that a11yCheck is asynchronous. Thus, you need to pass and invoke done function:

describe("a11y check", function() {
  it("has no accessbility violations (check console for errors)", function(done) {
    axe.a11yCheck($("#myElement")[0], function (results) {
        expect(results.violations.length).toBe(0);
        if (results.violations.length>0) {
            console.error(results.violations);
        }
        done();
    });
  });
});

Running aXe with QUnit 2.x

It is similar in QUnit. You also need to invoke done function, but first you need to get it by calling assert.async():

QUnit.test("a11y check", function(assert) {
    var done = assert.async();

    axe.a11yCheck($("#myElement")[0], function (results) {
        assert.strictEqual(results.violations.length, 0, "There should be no A11y violations (check console for errors)");
        if (results.violations.length>0) {
            console.error(results.violations);
        }
        done();
    });
});

Sample

I created a sample with button and input tag:

<div id="fixture">
  <button>My button</button> 
  <input type="text" />
</div>

This sample is not accessible because input tag does not have a label, and a11yCheck should report violations. The sample code, with Jasmine and QUnit tests, is available on github: axe-unittests.

Summary

While automated accessibility unit tests are great, you probably still want to use aXe plugin for Chrome to investigate reported violations. It’s more convenient, and it has neat user interface, while in unit tests you need to dig in into console errors.

When you start adding accessibility checks for different parts of your system, you may encounter many violations at first. It is better to still add tests, that ignore known violations, and then incrementally fix the issues. This approach prevents regressions, while delaying adding test until all violations are fixed may cause introducing new ones while fixing others.


Testing focus in JavaScript

This blog post is an overview of testing focus behavior in web browser.

During the work on Azure Portal I spent quite a bit of time on ensuring rich keyboard support. This requires appropriate focus management. When user press some keyboard shortcut, the focus should move to appropriate element. There is a good article on MDN about Keyboard-navigable JavaScript widgets. Focus behavior should be also tested very succinctly. As it is very easy to change (break) with even the smallest change in the HTML or JavaScript.

Standard test goes as follows:

  • Arrange: open some page
  • Act: execute some keyboard shortcut that should open particular page and/or set focus on particular element
  • Assert: check if expected element is focused

How to check if element is focused

The simplest way is to use jQuery:

expect($(element).is(':focus')).equals(true);

However this may not always work. Especially if you run your unit tests in parallel, because $element.is(‘:focus’) will not work when window does not have focus.

The better (right) way is to use document.activeElement:

expect(document.activeElement).equals(element);

This will work even when browser window is not focused.

Testing async actions

Sometimes, keyboard invoke asynchronous actions that eventually will change focus. This can cause:

  • false negative tests: assertion was executed before focus was eventually set
  • false positives: finally focused element, got focus after assertion was executed

The simplest recipe for the first problem is to delay assertion:

setTimeout(() =&gt; {
    expect(document.activeElement).equals(element);
    done();
}, 100);

The problem with this approach is choosing appropriate delay for assertion. Therefore it is better to avoid raw setTimeout, and use polling approach that I described in my post setTimeout considered harmful:

poll(
    () =&gt; document.activeElement === element, 
    () =&gt; {
	       assert(true);
	       start();
	  },
    () =&gt; {
	       assert(false);
	       start();
	  }
    );

The polling function can be also used for the second solution (by changing assertion order in callback functions). However, for the false positives problem, simple setTimeout is good enough, because we do not have a choice other than wait some particular period of time to execute assertion.

Invoking keyboard actions

There are 3 types of events that we can use to simulate keyboard action:

  • keydown
  • keyup
  • keypress

The safest bet is to use keydown. Why? An example might be using special keys. While in case of keypress – various browsers handle it differently (e.g., by not triggering the event), keydown is pretty consistent. I recommend you to check this article if you are interested in details. It was not updated for some time, but it will give you an idea.

In order to detect keyboard actions you may handle events on capture or bubbling phase. You should choose model depending on your needs, but in general bubbling phase is optimal solution. Additionally it is supported by jQuery, with browser inconsistencies taken care of for you.

To invoke keydown event in your tests, you need to set focus on the element first:

$(element)
  .focus()
  .trigger("keydown", { which: keyCode });

You can find JavaScript key codes here.

Summary

Use document.activeElement to check which element is focused.
Use polling approach for testing async actions.
Use keydown event, and bubble phase to invoke/handle keyboard actions.


Seattle Code Camp: Aurelia and TDD with TypeScript, AngularJS, and Node.js

Last Saturday I had a pleasure to speak at Seattle Code Camp. It was an amazing event, which was free. There were over 50 talks, 500 attendees, and raffle with many prizes, e.g., Microsoft Surface Pro 3!

Seattle Code Camp - Aurelia

Aurelia – the Next Generation JavaScript Framework you will love

Aurelia is a next generation JavaScript client framework that leverages simple conventions to empower your creativity created by Rob Eisenberg, creator of Durandal, who left Angular 2 team to create cleaner and simpler JavaScript framework of tomorrow written in ECMAScript 7!

In this talk I am showing how to get started with Aurelia, and how ES7 (aka ES 2016), JSPM, System.js, babel, and gulp can make you development amazingly fast, smooth, and enjoyable.

Video

Slides

Code

You can find demo app I created during presentation on github.

TDD with TypeScript, AngularJS, and Node.js

TypeScript helps to develop and maintain large web applications. Strong typing prevents from introducing bugs, but it doesn’t mean we can give up on testing. In this talk I showed how we can use TDD for building web applications with TypeScript on the client side (with AngularJS) and server side (with NodeJS). I showed a few JavaScript testing frameworks (mocha, Jasmine), and showed a few tools that can make Web Developer’s life easier (npm, gulp, bower, tsd, karma, protractor).

Video

Slides

Code

You can find demo app I created during presentation on github.

More is coming

I will be speaking about Aurelia, and TDD with TypeScript, AngularJS, and Node.js at various events this fall. You can find more details in my speaking page.


Regular Expression Translator

Another class I am taking this semester (except Real-Time Systems which I blogged about) is Formal Language Theory. It is very theoretical course. Most of the time is spend on formal proofs. Fortunately, one programming assignment was very fun.

The goal was to implement the program, which checks if one regular expression is subset of another. Thus, the input is two regular expressions.

RegEx AutomatasThe program use basic regular expressions notation and supports following operators:

  • + (union)
  • * (Kleene closure)
  • ε (Epsilon, empty expression)
  • 0,1,2 (literals)

To solve the problem, I had to take advantage of finite automatas (ε-NFA and DFA) in following order:

  1. Parse regular expressions E1 and E2
  2. Create ε-NFAs N1 and N2 using Visitor pattern and transition described here
  3. Convert ε-NFAs N1 and N2 to DFAs D1 and D2 based on this
  4. Create D’I DFA, which is intersection of D1 and D2 DFAs generated in step 3

Implementation details

Tools/Technologies:

  • Eclipse
  • Scala
  • Java
  • ANTLRWorks 2
  • Scala JUnit tests

I took advantage of ANTLR for parsing. I want to learn Scala and I am using it for every programming assignment in this semester if possible. ANTLRWorks 2 does not support Scala (it does not generate parser/lexer/visitor in Scala). However, ANTLRWorks 2 generates Java and interoperability between Scala and Java is easy. I decided to try it!

Scala is a main language in which this application was developed. ANTLRWorks 2 was used for parsing regular expression. The new version (v.2) generates Visitor class, which enables iterating through AST (Abstract Syntax Tree) nodes and perform appropriate actions. As I mentioned, ANTLRWorks 2 does not generate Scala code yet. Thus the generated parser and Visitor is in Java language.

During the intersection of DFAs creation, there is an exception thrown in 2 cases:

  • There exists some node of newly created D’I DFA, which is sum of D1 final node and D2 not final node
  • Some transition, which exists in D1 leads to dead node in D2

Exception contains string, which is accepted by L(E1), but not by L(E2). Then, this string is returned as a result of execution. If no exception was thrown, it means, every final state of D1 is also final state of D2. That means L(E1) ∈ L(E2) and ‘true’ is returned.

Running program

The main program is Main.scala. You can find there String variables defined with sample Regular Expressions. The easiest way is to replace them (if you want to test different ones). You can also read input from the console (by commenting and uncommenting appropriate lines).

Empty regular exception cause Exception throw.

There is also set of unit tests to test some regular expressions. You are more than welcome to add new Unit tests if you want to.

Input

Every closure has to have parenthesis: (exp)*
Union also require parenthesis and multiple unions are allowed: (exp1+exp2+exp3)
Sample inputs:

  • (0+1+2)*
  • ((0(0+2)*+1)*+2)
  • ((0(0+2)*+1)*+2+12+(00+21)*+0000110102)

Details about accepted input can be found in ANTLR grammar file.

Summary

In this assignment I was able to reuse my knowledge about compilers, learnt last semester in Compilers course I had. Additionally I had to implement automata representation and apply recursion to accomplish required tasks.

It is very simple translator. Handle only basic regular expressions and small language (only 0,1,2 literals allowed). However, the goal was to practice and understand how Regular Expressions work internally.

You can check the code of my translator on github: RegExpTranslator.


Compilers course I had

In the last semester (Fall 2013) I had a pleasure to take course CIS706 – Translator Design (aka Compilers) with Dr Robby at Kansas State University. It was great experience! I think it is the best course I have ever taken.

The way how this course is designed is just amazing. During course about compilers you will also learn many other useful things:

  • Java
  • Eclipse
  • Source Control (git)
  • Unit Testing (JUnit) and TDD
  • Reading legacy code
  • Java Byte Code

How? You have to build the compiler for simple, Java based language called ExtendedStaticJava. You have 4 homeworks: parsing, building AST (Abstract Syntax Tree), type checking and byte code generation. For each of them you have ‘legacy’ code, which implements compiler for StaticJava. Your task is to extend it to handle ExtendedStaticJava syntax. For each homework you get around 100 Unit Tests. Your grade depends on number of passing tests. If all test are passing you get 100%. Implementation does not matter…but you might need to pay for workarounds later, because each homework is based on the code from previous one. I experienced this. I made a few hacks in first homework and then I needed to fix them during homework 2. It cost me extra time and I had to send incomplete solution. It is exactly how it goes in “Real World”!

For final project (last month of semester) we needed to create ExtendedStaticJava to C transpiler with Garbage Collector. Transpiler is source-to-source compiler. In other words: resulting code was C with Garbage Collector calls (GC was also implemented by us in C). We reused our homeworks (without byte code generation part) and generated C code (with GC calls) using StringTemplate.

Let’s take a look how all above help us to be better developers.

Most of students probably already know Java. If not – this course is the best time to learn. Java and C# are the most popular, static typing programming languages (today). When you learn one of them – it is easy to catch up with another. Java is still the most popular language in case of job offers (source):

traditional Language Job Trends - Indeed

Thus, probability that you will be programming in Java after you graduate is pretty high. When you will be programming in Java, you will be probably using Eclipse – its most popular IDE. Another ‘standard’ in industry is Source Control. If the company you are working for is not using it – you should change your company(!), because something wrong is going on there. If you are not familiar with git (leader in Open Source community, now being adapted by the industry) then this course is the best time to learn. Nowadays, even big companies (e.g. Microsoft) open sourcing their code (e.g. ASP.NET or SignalR), which is available through git.

Unit testing is very popular nowadays. Lots of commercial projects use this technique to ensure software quality and development stability. Many projects are also using TDD (Test Driven Development) approach. First your write the test (as the requirement) and then code. When test pass, you refactor the code. During the course we were using JUnit (for now: No.1 Unit Test Framework for Java). Additionally, when you learn how to work with Unit Tests, then picking up a different framework is very easy (even in different language).

Software Development is a team work. Because of that, you will need to read code of other developers everyday. In most of courses during your University career you need to create some applications from scratch. Sometimes you are working in team of 2 or 3 people. Then in your first job, you get 100 000 lines of code written by somebody else. Often without documentation and sometimes people who wrote this code are not working there anymore. In other words – you’re screwed. Compilers class will prepare you for that. During the course you will need to understand the existing code (StaticJava compiler), extend it and introduce some changes to get ExtendedStaticJava compiler working and Unit Tests passing.

Besides learning Java, you will also learn how JVM (Java Virtual Machine) and ByteCode are working. The idea of intermediate code is also used in C#: code is compiled to an Intermediate Language (IL) which then runs in the Common Language Runtime (CLR). It is very helpful for understanding how high-level code is translated to Assembly language.

It is also worth to mention about the essence of the course: compilers. The course is based on the popular Dragon Book. You will learn about every stage of compiling: lexical analysis, parsing, creating AST (Abstract Syntax Tree), semantic analysis, optimization and code generation. For parsing and AST creation you will use ANTLR, which is the most popular tool for such purpose nowadays. Finally you will get understanding, what is going on between your high-level code and electrical impulses flowing in your computer.

Even if you know all of these things (Java, Eclipse, git, Unit testing) it is still worth to take this class. You will apply your knowledge in practice and learn about compilers. This knowledge is useful not only to understand how computers work, but also to learn about algorithms and data structures, which are applied there.

You can find more information about CIS706 on compilers website. Moreover the StaticJava code is available on github: github.com/santoslab/compiler. There is also nice Compilers Course by Stanford University on Coursera, which can be used as supplement for this class.