Simple Test Plan for a webpage
The sooner you start testing a webpage design for usability, the more you are going to save in development and maintenance costs:
The rule of thumb in many usability-aware organizations is that the cost-benefit ratio for usability is $1:$10-$100. Once a system is in development, correcting a problem costs 10 times as much as fixing the same problem in design. If the system has been released, it costs 100 times as much relative to fixing in design.
– (Gilb, 1988)
- The ideal number of users for an inexpensive test plan is 2-5, according to usability consultant Jacob Nielsen (in his article testing_two_users).
- As with any test plan, it is important that you decide what is to be tested before you conduct the test, though you can improvise as you go along.
- It is important that the user is observed, and his or her difficulties are noted. But you, the tester, should resist the temptation to prompt them or explain to them how to do something if they are having difficulty. You need to allow your users to muddle through and make mistakes.
- Make it clear to your users that you are testing the webpage, and not them. Reassure them that if they make mistakes, it is going to be very helpful to you.
- You need to ask each user to make an effort to speak all their thoughts out loud while they are viewing the webpage. You may need to remind them to do this as you go along.
Sometimes it is very enlightening to learn that a user cannot complete your test at all because of some fundamental lack of understanding or because of a ‘stupid’ mistake he or she makes. Be grateful if this happens because you have just learned a valuable lesson. Remember to be kind to your user, and make it clear you do not think they are stupid. In this case, you might want to go and modify your design immediately, and start retesting.
Purpose of the Test
Define the purpose of your test, for example to test the usability of a particular webpage, or to test the functionality of an online form.
Note the the official purpose of the webpage which is to be tested.
Find a cross-section of users who represent your major audience groups. For example, for a website about content-writing, the target audience groups might include:
- Website owners
- Content writers (i.e. people who have contributed to the content of some websites)
- Content managers (people responsible for managing the content on corporate websites)
- Freelance writers
Impressions to be captured
- The all-important first impressions of the webpage
- The benefits users receive from reading your content or completing your form, etc.
- How difficult or easy it is for the users to complete their assigned tasks
- Particular likes or dislikes about the webpage
Each user is :-
- Shown the webpage on their computer screen. Alternatively, they can be given a printed copy of the webpage
- Given some tasks to complete and questions to answer
- One of the tasks might be for the users to first find the test page from the home page, then to process or read it.
You may ask your users to imagine themselves in a particular situation, for example, they are going on holidays and want to make a hotel reservation in a particular city.
Example Questions asked
Prepare a list of questions for your users to answer. For example:
Give the user a particular length of time to get a first impression of the webpage, for example 15 seconds only, then ask:
- What was your first impressions of the webpage?
Here as some example questions to ask your user after they have had sufficient time to read your webpage content:
- What do you understand the purpose of the webpage to be? Was this purpose clear from the beginning?
- How easy was the content to digest?
- Was the length of the article daunting? Note how long did it takes them to read your article or complete your tasks
- What do you understand by the term “…”? (Pick a term that is explained in your article)
- On a scale of 1 to 10, how helpful was this article in educating you about the importance of “…”? (Ask a question related to your content)
- Who do you think would benefit from reading this article What benefits would they gain?
- What was your level of confidence – (a) at the beginning and (b) at the end of reading the article?
- Is the content credible?
- Do you trust the content author?
- What did you like about the webpage?
- What did you dislike about the webpage?
- Did you experience any difficulties on the webpage?
- What was the overall quality of your experience? How did you feel at the end of the exercise?
- What could be done to improve the webpage content or presentation?
- Which links would you like to follow to read more?
- Do you have any other comments or suggestions?
Online User Testing services
There are online services, such as www.usertesting.com, where you can pay for video reviews of your web pages on either desktops or mobiles. The cost is US$49 per video (as at January 2016). This is well worth budgeting for as part of your website development. You can define the job, select your users, give them tasks and questions to answer. The users record their thoughts out loud as they view your page, and provide a 15 minute video. In my experience, you need at least 3 user tests in order to get a balance of different viewpoints. I have found I always learn something of value from each test – especially when I discover the stumbling blocks and difficulties the user has. Typical questions I ask are:-
- What frustrated you most about this site?
- If you had a magic wand, how would you improve this site?
- What did you like about the site?
- How likely are you to recommend this site to a friend or colleague (0=Not at all likely, and 10=Very Likely)?
You may want to order some tests to be run on desktops, then another group of tests to be run on smartphones. The mobile user experience can be quite different from the desktop experience, and so you may want to ask different questions to assess the quality of the mobile user experience.
In Steve Krug’s book, Don’t Make Me Think, he has a chapter on Usability Testing with some good tips in it, and a script of an imaginary session (what the tester says, what the user says). This book is for light reading.