Monthly Archives: August 2015

Speech Recognition in the Browser

Last Thursday I had a pleasure to give talk about Speech Recognition in the Browser at the Code Fellows in Seattle.

Many people were surprised how easy it is to add speech recognition to your website with pure JavaScript. So I thought I will share a few code snippets here. It works in Chrome only so far.

Recognizing speech

This is how you can translate speech to text:

var sr = new webkitSpeechRecognition();
sr.onresult = function (evt) {
    console.log(evt.results[0][0].transcript);
}
sr.start();

You can also get the confidence level of the result:

var sr = new webkitSpeechRecognition();
sr.onresult = function (evt) {
    console.log(evt.results[0][0].transcript, evt.results[0][0].confidence);
}
sr.start();

You can get interim results:

sr.interimResults = true;	// false by default
sr.onresult = function(evt) {
	for (var i = 0; i < evt.results.length; ++i) {
		console.log(evt.results[i][0].transcript);
	};
};

Or different alternatives of recognized speech:

sr.maxAlternatives = 10;	// default = 1
sr.onresult = function(evt) {
	for (var i = 0; i < evt.results[0].length; ++i) {
		console.log(evt.results[0][i].transcript);
	}
}

You can set a language, e.g., to Polish:

sr.lang = 'pl-PL'

All above will stop recognition when you stop speaking. In order to do not stop recognition you need to set continuous flag to true. Additionally, this will treat every fragment of you speech as interim result, so you need to update onresult callback too:

sr.continuous = true;	// false by default
sr.onresult = function(evt) {
	console.log(evt.results[evt.results.length-1][0].transcript);
};

Speech Recognition object has other callbacks (than onresult) that you can take advantage of:

sr.onstart = function() { console.log("onstart"); }
sr.onend = function() { console.log("onend"); }
sr.onspeechstart = function() { console.info("speech start"); }
sr.onspeechend = function() { console.info("speech end"); }

Emitting speech

var msg = new SpeechSynthesisUtterance('Hi, I\'m Jakub!');
speechSynthesis.speak(msg);

You can also change the speaker voice:

var voices = window.speechSynthesis.getVoices();
msg.voice = voices[10]; // Note: some voices don't support altering params

There is also other options you can set:

msg.volume = 1; // 0 to 1
msg.pitch = 2; //0 to 2
msg.text = 'Hello World';
msg.lang = 'en-US';

msg.onend = function(e) {
	console.log('Finished in ' + event.elapsedTime + ' seconds.');
};

Summary

Speech is coming to the browser, and you can not stop it. The question is when most of websites will add voice support. Check out voiceCmdr – a library that I blogged about earlier this year, which helps to add voice commands to your websites in very easy way. You can also check out website that can be navigated with voice commands – you can find available commands in my blog post. You can find entire logic for voice commands support in this file (lines: 38-103).


setTimeout considered harmful

setTimeout - Steve's Tweet

Recently I learned the hard way about setTimeouts side effects.

101 setTimeout issue

Let’s say we have a following piece of code:

var someVar = 0;

var changeVar = function () {
    someVar = 10;
};

And unit test:

QUnit.asyncTest('should change variable to 10', function () {
    // Arrange
    someVar = 0;

    // Act
    changeVar();

    // Assert
    equals(someVar, 10);
});

Of course test passes, and we are happy.

Then, after some time, because of a reason (e.g., we want to fix some bug), we are adding setTimeout to changeVar function:

var changeVar = function () {
    setTimeout(function () {
        someVar = 10;
    }, 10);
};

Unit test does not pass anymore. So we are adding setTimeout to our unit test as well (ideally: appropriately longer than one in changeVar function to avoid confusion!):

QUnit.asyncTest('should change variable to 10', function () {
    // Arrange
    someVar = 0;

    // Act
    changeVar();

    // Assert
    setTimeout(function () {
        equals(someVar, 10);
    }, 20);
});

Then, we are introducing change to our code. Update only when someVar is not zero. We update function, and test accordingly:

var changeVar = function () {
    setTimeout(function () {
        if (someVar !== 0) {
            someVar = 10;
        }
    }, 10);
};

QUnit.asyncTest('should change variable to 10 if someVar != 0', function () {
    // Arrange
    someVar = 0;

    // Act
    changeVar();

    // Assert
    setTimeout(function () {
        equal(someVar, 0);
    }, 20);
});

Everything works. Great! Then, somebody else is fixing another bug – of course by increasing timeout:

var changeVar = function () {
    setTimeout(function () {
        if (someVar !== 0) {
            someVar = 10;
        }
    }, 50);
};

Still works, but when after some time we decide to change our logic:

var changeVar = function () {
    setTimeout(function () {
        if (someVar === 0) {
            someVar = 10;
        }
    }, 50);
};

Our test is still passing…when it shouldn’t!

Of course all of this is happening in large codebase with more complex logic.

But this is not that bad. Let’s take a look at more interesting scenario.

More complex case

We are in worst situation when we have waterfall of setTimeouts.

var fun1 = function () {
    setTimeout(function () {
        someVar++;
    }, 10);
};

var fun2 = function () {
    setTimeout(function () {
        fun1();
        someVar = 10;
    }, 10);
};

Guess if this unit test will pass:

QUnit.asyncTest('should change variable to 11', function() {
    // Arrange
    someVar = 0;

    // Act
    fun2();

    // Assert
    setTimeout(function () {
    	equal(someVar, 11);
    	start();
    }, 20);
});

You think it should? You are wrong!

But this test will pass:

QUnit.asyncTest('should change variable to 11', function() {
    // Arrange
    someVar = 0;

    // Act
    fun2();

    // Assert
    setTimeout(function () {
    	equal(someVar, 11);
    	start();
    }, 21);
});

Do you see the difference? Yes, timeout is 21 instead of 20. And of course everything will fall apart if somebody increase timeout in fun1 or fun2.

Solution

The best solution is not to use setTimeout at all. Unfortunately we need async operations sometimes. In this case – you should use promises if possible. Unfortunately World is not perfect, especially Web Development World, and sometimes you have to use setTimeout (e.g., for UI effects etc.). You may think that if you set long enough timeout in your unit tests, everything should be good, right? Well…remember that it will make your unit tests slower. Instead – you should use polling approach.

To apply it to the last example – using “Without Deferreds” approach – copy poll function to your codebase:

function poll(fn, callback, errback, timeout, interval) {
    var endTime = Number(new Date()) + (timeout || 2000);
    interval = interval || 100;

    (function p() {
            // If the condition is met, we're done! 
            if(fn()) {
                callback();
            }
            // If the condition isn't met but the timeout hasn't elapsed, go again
            else if (Number(new Date()) < endTime) {
                setTimeout(p, interval);
            }
            // Didn't match and too much time, reject!
            else {
                errback(new Error('timed out for ' + fn + ': ' + arguments));
            }
    })();
}

And call it in your unit test:

QUnit.asyncTest('should change variable to 11', function() {
    // Arrange
    someVar = 0;

    // Act
    fun2();

    // Assert
    poll(
	    function() {
	        return someVar === 11;
	    },
	    function() {
	        ok(true);
	        start();
	    },
	    function() {
	        ok(false);
	        start();
	    }
	);
});

Now, you do not have to guess what value will be appropriate for setTimeout delay, and you will speed up your tests as well.

When I see some strange behavior in code, the first thing I am looking at are setTimeout calls.