“If you don’t have oodles of usability people, up front design by a good designer provides a lot more bang for buck than using that same designer to do usability tests” says Red Hat’s Seth Nickell.
Seth Nickell on Usability Testing
About The Author
Eugenia Loli
Ex-programmer, ex-editor in chief at OSNews.com, now a visual artist/filmmaker.
Follow me on Twitter @EugeniaLoli
“Many designers advocate usability testing less because it improves the design, and more because its a useful tool for convincing reluctant engineers that they need to listen: usability testing sounds all scientific. Usability testing can be a very useful technique for trying to get improvements implemented in a “design hostile environment”.”
That says it right there, and it’s not just reluctant engineers but bosses as well, for there’s a lot of “design hostile environments” out there. OSS and closed source.
Disclaimer: I am not a usability expert.
Seth’s example of using usability tests to find the better interface seems wrong-headed to me. Both interfaces A and B gave some users serious problems. This suggests that both are broken by design, and need re-desiging and re-testing. I guess I’m just reinforcing his point that usability testing cannot be used this way.
Also of interest is Jakob Nielsen’s article “Why You Only Need to Test With 5 Users,” http://www.useit.com/alertbox/20000319.html His point is that the law of diminishing returns kicks in after 5 users have been tested, so many small tests gives you more bang for your buck than a single large test.
I’m presuming that by “usability testing” Seth refers to “user testing”. Interfaces can (and are) tested using empirical techniques without reference to a user population (see the various GOMS techniques, ACT-R, TAG etc), though user testing can be performed to “validate” the results.
“Usability testing” does not test just “learnability”. For example, errors arising from habituation can be measured.
RE: Matthew Bishop. Interesting article, but Neilsens work isn’t held in very high esteem within the HCI community. See ‘The “Magic Number 5”: Is it enough for web testing’ by Barnum (HCI 2002) for a good criticism of methods using small populations (this article focuses on web design: application design is often more complicated as it presents a wider rank of tasks than web pages which are largely information location tasks). Neilsen tends to concentrate upon business friendly ideas (stuff like ten bullet points to improve your website that PHB’s adore) that don’t get much thrift in the professional HCI community.
Having said this, Seth does make a very important point: chucking a design at a set of users is not the be-all and end-all of HCI by any means, rather it is just one important tool.
This reminds me of another thing on Mr. Nickell’s website. From December 29: Best design principle I’ve heard in a while: “What can we fit on the form?” Unfortunately, too many user interfaces are designed that way. (I should know, I’m the one who told him that design principle.) Basically, when I design a bad user interface, it’s because I got lazy. I know I should spend time up front designing the program right, but I get impatient and just start coding. Then about halfway through development I realize “Oops, this is a bad UI design” but by that point it’s too late and I just have to make do.
Now, with help from my coworkers we usually get the underlying data model pretty robust, because that kind of designing is fun for us as developers. But none of us are particularly good at user interface design, and none of us like it, so we tend to forget about it or put it off until later.
Probably what we at my company should do is designate someone as the UI designer. He will design the UI while the others work on the data model and the underlying code. But dang it, I don’t want to work on UI stuff, it’s hard and boring. And my attitude is exactly why software so often turns out bad.
Read the entry for yourself at: http://www.gnome.org/~seth/blog/2003/Dec
Err, just to clear something up: I do not work with Seth Nickell, nor do I work for his employer, Red Hat. My post kind of implied that, but I didn’t mean to make it sound that way. But I wish I did work for Red Hat. (Well, sort of. I do like my current job, but I don’t get to write Free or Open Source software.)
i work at the world’s busiest biggest site (guess) and we are currently trapped going down the rabbit hole of analytics, user-testing…anything to avoid creative thought.
as seth points out – this stuff is useful for dotting i’s, crossing t’s, but too many managers are trying to build a business by the numbers presuming that is the path of minimized risk. it is in fact the path the maximized mediocrity.
I think it’s rather ironic to hear a Red Hat guy talking about proper design and usability. He might be right, but you’d never know it from Red Hat’s product line.
Can you please email me or get in touch with me?
Thanks, it’s pretty important…
My email is:
[email protected]
And, my apologies for being off-topic as well.