Lies, damn lies, and usability metrics

I like Jakob Nielsen. I like his drive and passion for great usability. But he does produce some severely dodgy statistics from time to time. This week, in his Alertbox column, he headlines with the quote:

“Software has great potential for getting better, as shown by an under-appreciated feature in Windows XP that can save users $2,000 per year.”

In the article he explains how he gets at this figure. I won’t duplicate the calculation here, but the heart of it is the assertion that a 10% increase in reading speed (by using Cleartype) results in a 10% increase in productivity.

Sorry Jakob; doesn’t follow. A 10% increase in reading speed means that you’ve got 10% more time to spend nattering with your colleagues over coffee.

A 50% increase in reading speed, now that would be useful. But I doubt very much if the productivity gain would be anywhere near that. Probably closer to 10-20%. The main reason for this is that hardly anyone spends all their time reading continuously. Most of the time you read a short chunk, then do something else. And the time it takes to go from reading to something else will swallow up 10% with ease. Joel Spolsky writes about exactly this in his article on task switching.