Usability sessions: the online attempt
April 20th, 2009
We experienced already a number of times that usability testing is absolutely essential to create good software applications. If you haven’t bought Steven Krug’s “Don’t make me think‘ book, stop reading this post and order it. It’ll be your best buy this decade.
We are going through a redesign of our website and the screens. All we have is a static website, and a number of screen impressions (in PDF format). We have put up a question on a number of places. In this post, I’d like to share our experience about how that went.
“Linked in” is king
Of all places we have posted our invitation to validate our screens, LinkedIn.com was definitively the most successful. We got over 10 answers of people that really seem interested. “Yahoo Answers” was a distant second: only 1 answer on the 3rd day. All other places (blog posts, develop forums…) gave 0 results. Zippo, nada, nieltsj (I just invented that one).
Free cookie
As part of our offer, we gave away free live time subscriptions. What did we notice?
- Nobody bothered to ask us what and how this FREE thing was going to happen. To be honest, we don’t yet know ourselves.
- When I asked those who participated if they really wanted the free subscription, they all confirmed.
- We were in no position to do a split test on this one, but from the reactions we got, I think ‘FREE’ at least drew some extra attention.
50% success rate
Of all people that initially voiced their wish to participate, only half actually replied. I didn’t push people to participate.
Be specific about the format
One lady said she was disappointed by the format of the test. She expected a functional prototype, not some static PDF thing. I guess we should have made this clearer at the start.
Magical answers
I monitored my inbox for days to get the feedback. When results started pouring in, we were truly amazed:
- the detail was phenomenal,
- the analysis was thorough and
- the similarities between all participants was striking.
Many people went over each screen in great detail, giving pro’s and con’s about various aspects of the design.
Conclusion
The online version of usability testing certainly misses out on some of the finer nuances of people’s reaction and initial confusion/enlightenment on your screens. However, when on the very early phases of design, I would compare this approach to the real live equivalent of ‘hallway testing‘. It gives you immediate feedback you just can’t ignore. So all participants certainly will get their FREE-Forever ticket.