Jakob Nielsen: Tot 40% van de tijd bij usability-onderzoeken gaat verloren aan het uitvoeren van onbelangrijke activiteiten door de gebruikers.

Tijdverslinders

Starting the session with an extensive demographic survey. It's better to collect this data using an online website survey. For test participants, make do with the data collected during the up-front screening process and refrain from asking additional questions during the test.

Asking users for subjective satisfaction ratings after each task. Subjective ratings are weak data in the first place. Research projects aside, overly fine-grained ratings are rarely worth the time required to collect them.

Using a satisfaction questionnaire with dozens of questions instead of a single overall satisfaction score. It's stupid to ask users to rate, for example, how much they like the graphics. If people have strong feelings about how something looks -- whether it be pleasing, ugly, or inappropriate -- they'll voice those feelings during the task. The one thing a questionnaire should ask users about is overall satisfaction. Detailed issues are much more valid if you assess them based on users' behavior while performing tasks, rather than asking for a retrospective rating.

Ending the session with a long discussion about how users might feel about potential new product developments. Again, focus groups are better for this. Also, users' reactions to prototype designs while performing tasks are much more valid than people's hypothetical speculations about what they might like. Spend your time collecting valid data rather than speculative data, even if doing so requires you to mock up a few more pages.

» Jakob Nielsen's Alertbox, September 12, 2005