Welcome to Screen Space your podcast about creating usable, accessible, effective, and efficient web, blog, and digital media design for the everyday (and non-expert) designer. This is episode 12 of Screen Space “Usability & Usability Testing 101 Part 2—Selecting Users.”
In this episode, I discuss the first part of usability testing—user analysis and selection. I will talk about determining who your users are going to be for your testing and deciding how many users to test. There will be about four more parts to this series, walking you through the steps to conducting and analyzing usability testing of your web, blog, or digital media. Go back to Screen Space 11: Usability & Usability Testing 101 for my discussion on what usability is, a definition of usability testing, and an outline of the steps to conducting usability test. You may also find Screen Space 10 on User-Centered Design helpful too.
I am your host, Dr. Jennifer L. Bowie. I teach and conduct research in areas related to digital media, web, and blog design. To start with, welcome new listeners from Melbourne, Australia; Abuja, Nigeria; and Ottawa, Canada! Enjoy, design well, and let me know if there is anything you would like me to cover.
In this episode, I will cover determining your users, creating user profiles, choosing user profiles to test, and deciding how many users to test. I will continue working in this episode with the same example I introduced in episode 11 to illustrate how to apply the steps of usability testing—imagine you have a photography blog with a decent sized audience. You want to get more users and see how usable the blog is for your current users. So, let’s get going into Part 2—Selecting Users.
Determine the users
First, you need to determine who the users of your web, blog, or digital media are. Ask yourself who your actual users are. If you can, actually find this out through surveys, analytics, and other methods. If not, think about who you are writing to. Conduct a basic user analysis. It is possible you have more than one groups of users. If so, determine your various user types. You may find it helpful to consider a variety of typical user categories to determine your users such as:
- Demographics: Consider age, sex, race, educational level, cultural background, socioeconomic status, and other demographic information. From this create demographic profiles of your users or user groups. This will allow you to visualize your audience.
- Experience level: Analyze user experience levels with the product, with products of the same genre, and with required technology. This will allow you to test users who have a similar experience backgrounds as your user groups.
- Use types and needs: Determine what types of use and needs bring your users to your site, blog, or other media. Users who are coming for general information may be quite different than users who are interested in making purchase or leaving a review. What do they need from the site? Why are they there? Determining the use types and needs will allow you to further differentiate users and help with later steps of testing.
- Other areas: Consider also a variety of other user differences and possibilities such as:
- Motivation: Are they motivated? Do they want to be there? Do they want to be doing what they are? Buying concert tickets leads to much different motivation levels than paying taxes, at least for most people.
- Learning style: What are their learning styles? For example, are they visual learners? Auditory learners?
- Subject matter knowledge: How much do they know about the subject matter of your site?
- Location of use: Where do they use your site, blog, media? Home? Work? On a laptop? Tablet? Desktop? Phone? On the go?
- Physical characteristics: Do the users have any physical characteristics that may impact use? This may be a more valid determination for a physical product, but still consider how the body may interact with your site and what physical characteristics your users have that matter to this interaction.
- Disabilities or impairments: Do consider any disabilities or impairments your users may have from color blindness and learning disabilities to more severe disabilities.
Once you have considered these various areas, you should be prepared to move onto the next step, creating user profiles.
Create user profiles
To create your user profiles you need to break your users into clear subgroups, likely based on the differences you found when considering demographics, experience level, use types and needs, and other areas. You then need to profile/define the characteristics of each subgroup.
For the photography blog you may find you have two general audiences:
- Group 1 “Fans”: Middle-aged, middle class, American users with limited photography experience and a love of art. They come to the blog to look at photos, get tips on taking better photos with their digital point-and-shoot cameras, and possibly buy or download some of your photos. These people are fans of your work, thus the group name.
- Group 2 “photographers”: A slightly younger group of users with more tech savvy and photography experience who are amateur photographers themselves. Some have their own photography blogs. They tend have household incomes ranging from $50,000 to $150,000 and are likely to spend money on technology and photography equipment. They come to the site to see what other photographers are doing, to build the photography community and support your work, and to get and share more advanced photography tips.
You may also want to develop personas for each group. Personas are fictional characters that represent a user group. These make it easier to picture and consider your audience and allow a deeper focus on the users’ needs and goals. A good persona has a name, picture, and background information. I’ll discuss personas in detail in a later episode and will link to a decent short article on personas in the transcript for more information. (URL: http://www.usability.gov/methods/analyze_current/personas.html).
Choose user profiles to test
Once you have your user profiles, you need to choose which user profiles you will test. Ideally you will test users from all major profiles—in this case the fans and photographers. However, some sites have a huge number of user profiles or there is limited time or funds, and it may not be possible to test each profile. In that case, choose profiles based on highest number of users in that profile or profiles that you think may have the greatest usability issues. As an everyday and non-expert designer, you may not even be able to do this. If the photography blog is simply a hobby, you may need to use what is called “convenience sampling”—select users (or even non users) that you have access to, even if they are not users from your profiles. You may test using your spouse, a parent, the babysitter you hire to watch your kids, a co-worker, and some friends. Even if these are not ideal users, still test. You can learn some very important information on how your site, blog, or media really works simply from watching people use it.
I recently had someone test her grandparent on a site for a much younger audience, but she got some vital information from watching her grandmother attempt the usability tasks. She used the information from the testing to redesign a much more usable site for all her users (including her newest user—her grandmother).
Decide how many users to test
The last step in the selecting users part of the testing process is to decide how many users to test. This may seem like an easy part, but this is a hotly debated topic in usability testing. Jacob Nielsen, a web usability guru, wrote a well-cited article “Why You Only Need to Test with 5 Users” (http://www.useit.com/alertbox/20000319.html). Since this article, the number five is often given as the answer to “how many users should I test?” But anyone who suggests five has not read the whole article. Nielsen suggests five users if you are doing iterative testing and he suggest three iterations, which would be an ending result of 15 users. Based on his research, he found that 15 users will give you almost 100% of the usability problems. He also suggests fewer users from your user profile per iteration if you are testing different groups of users, thus ending up with a similar overall number of users or even more. Due to the misunderstanding of Nielsen’s message on the number of users, many other usability researchers have suggested and researched other numbers. For instance, on page 382 Faulkner suggests that 20 users “to approach increasing levels of certainty that high percentages of existing usability problems have been found in testing” as she discovered 20 users found 95% of the usability problems. Jared Spool and Will Schroeder also studied the problem of the number of users and based on their results project 90 users may be necessary.
I would not suggest going so high as 90 users unless you have a very large and complex site, blog, or media. For most websites, blogs, and media, I would suggest Nielsen’s iterative testing with 5 users and three iterative tests, for a resulting 15 or so users. But for more casual websites, blogs, and digital media, like you may be working on, I suggest any number of users you can manage. Even testing with a single user is helpful. Nielsen found a single user will often uncover almost one-third of the usability problems. A few users is great—3 gets you about two-thirds of the problems and 5 gets you 80% of the problems. So, don’t worry about the number of users—any testing is helpful.
And that is how you select users for your usability testing. To summarize: first, determine your users are; then create user profiles of your major user groups, next choose user profiles to test, and finally decide how many users to test. Remember, a single user is 33% better than none.
That wraps up “Usability & Usability Testing 101 Part 2—Selecting Users”. Join me next week for the first of the Computers and Writing interviews. We will start with Lars Söderlund’s interview on “Gradual Growth Web Design: Guides, Regularity, and Product (or “Guns Ruin Picnics”). In two weeks, we’ll hear from Sarah Brown on “Personal Branding and Online Identity Construction.” I will return to the “Usability & Usability Testing 101” series after the four interviews.
If you have questions, comments, or thoughts on what you want me to cover please send me an email at firstname.lastname@example.org or check out the Screen Space blog—www.screenspace.org. You can also follow Screen_Space on Twitter for hints, tips, advice, news, and information on designing websites, blogs, and other digital media texts. Also, check out the blog for a transcript of this podcast complete with links and resources.
Have fun and design well!
Screen Space is licensed under a Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License. So, feel to send a copy to all your users, but don’t change the podcast, do give me and Screen Space credit, and don’t make any money off of it.
Screen Space’s opening music today is “African Dance” by Apa Ya off of Headroom Project and the closing music is “Survival” by Beth Quist off of “Shall We Dance”. Both these selections are available from Magnatune.
Episode 12 Links and References:
Past Screen Spaces podcasts you may want to refer to:
- Screen Space 11: Usability & Usability Testing 101 http://screenspace.org/?p=147
- Screen Space 10: User-Centered Design 101 (Why user-friendly is not enough) http://screenspace.org/?p=115
The Persona’s resource:
“Develop Personas” on Usability.gov. http://www.usability.gov/methods/analyze_current/personas.html
References on number of users to test:
- Jacob Nielsen, “Why You Only Need to Test with 5 Users.” Useit.com. http://www.useit.com/alertbox/20000319.html
- Lisa Faulkner. “Beyond the five-user assumption: Benefits of increased sample sizes in usability testing.” Behavior Research Methods, Instruments, & Computers, 35 (3), 379-383. http://usableconnections.com/papers/Faulkner_BRMIC_Vol35.pdf
- Jared Spool and Will Schroeder. “Testing web sites: five users is nowhere near enough.” Proceedings of CHI EA ’01. http://portal.acm.org/citation.cfm?id=634236