Simple Test Plan for a webpage

simple test planHere is an example of a very simple test plan for a webpage.   The test can be conducted in front of a computer screen, or over a piece of paper (a printed copy of the webpage) in a coffee shop.

The sooner you start testing a webpage design for usability, the more you are going to save in development and maintenance costs:

The rule of thumb in many usability-aware organizations is that the cost-benefit ratio for usability is $1:$10-$100. Once a system is in development, correcting a problem costs 10 times as much as fixing the same problem in design. If the system has been released, it costs 100 times as much relative to fixing in design.
– (Gilb, 1988)

Test Guidelines

  • The ideal number of users for an inexpensive test plan is 2-5, according to usability consultant Jacob Nielsen (in his article testing_two_users).
  • As with any test plan, it is important that you decide what is to be tested before you conduct the test, though you can improvise as you go along.
  • It is important that the user is observed, and his or her difficulties are noted.  But you, the tester, should resist the temptation to prompt them or explain to them how to do something if they are having difficulty.  You need to allow your users to muddle through and make mistakes.
  • Make it clear to your users that you are testing the webpage, and not them.  Reassure them that if they make mistakes, it is going to be very helpful to you.
  • You need to ask each user to make an effort to speak all their thoughts out loud while they are viewing the webpage.  You may need to remind them to do this as you go along.

Sometimes it is very enlightening to learn that a user cannot complete your test at all because of some fundamental lack of understanding or because of a ‘stupid’ mistake he or she makes.  Be grateful if this happens because you have just learned a valuable lesson.  Remember to be kind to your user, and make it clear you do not think they are stupid.  In this case, you might want to go and modify your design immediately, and start retesting.

Purpose of the Test

Define the purpose of your test, for example to test the usability of a particular webpage, or to test the functionality of an online form.

Note the the official purpose of the webpage which is to be tested.

User demographics

Find a cross-section of users who represent your major audience groups.  For example, for a website about content-writing, the target audience groups might include:

  • Website owners
  • Content writers (i.e. people who have contributed to the content of some websites)
  • Content managers (people responsible for managing the content on corporate websites)
  • Freelance writers

Impressions to be captured

  • The all-important first impressions of the webpage
  • The benefits users receive from reading your content or completing your form, etc.
  • How difficult or easy it is for the users to complete their assigned tasks
  • Particular likes or dislikes about the webpage

Task assigned

Each user is :-

  1. Shown the webpage on their computer screen.  Alternatively, they can be given a printed copy of the webpage
  2. Given some tasks to complete and questions to answer
  3. One of the tasks might be for the users to first find the test page from the home page, then to process or read it.

You may ask your users to imagine themselves in a particular situation, for example, they are going on holidays and want to make a hotel reservation in a particular city.

Example Questions asked

Prepare a list of questions for your users to answer.  For example:

Give the user a particular length of time to get a first impression of the webpage, for example 15 seconds only, then ask:

  • What was your first impressions of the webpage?

Here as some example questions to ask your user after they have had sufficient time to read your webpage content:

  1. What do you understand the purpose of the webpage to be? Was this purpose clear from the beginning?
  2. How easy was the content to digest?
  3. Was the length of the article daunting? Note how long did it takes them to read your article or complete your tasks
  4. What do you understand by the term “…”? (Pick a term that is explained in your article)
  5. On a scale of 1 to 10, how helpful was this article in educating you about the importance of “…”? (Ask a question related to your content)
  6. Who do you think would benefit from reading this article  What benefits would they gain?
  7. What was your level of confidence – (a) at the beginning and (b) at the end of reading the article?
  8. Is the content credible?
  9. Do you trust the content author?
  10. What did you like about the webpage?
  11. What did you dislike about the webpage?
  12. Did you experience any difficulties on the webpage?
  13. What was the overall quality of your experience?  How did you feel at the end of the exercise?
  14. What could be done to improve the webpage content or presentation?
  15. Which links would you like to follow to read more?
  16. Do you have any other comments or suggestions?

After each interview with a user, you may decide to revise your test-plan because you learned something from the experience, and that is good.

Online User Testing services

There are online services, such as www.usertesting.com, where you can pay for video reviews of your web pages on either desktops or mobiles.  The cost is US$49 per video (as at January 2016).  This is well worth budgeting for as part of your website development. You can define the job, select your users, give them tasks and questions to answer.  The users record their thoughts out loud as they view your page, and provide a 15 minute video.   In my experience, you need at least 3 user tests in order to get a balance of different viewpoints.  I have found I always learn something of value from each test – especially when I discover the stumbling blocks and difficulties the user has.  Typical questions I ask are:-

  1. What frustrated you most about this site?
  2. If you had a magic wand, how would you improve this site?
  3. What did you like about the site?
  4. How likely are you to recommend this site to a friend or colleague (0=Not at all likely, and 10=Very Likely)?

You may want to order some tests to be run on desktops, then another group of tests to be run on smartphones. The mobile user experience can be quite different from the desktop experience, and so you may want to ask different questions to assess the quality of the mobile user experience.


Recommended reading:

In Steve Krug’s book, Don’t Make Me Think, he has a chapter on Usability Testing with some good tips in it, and a script of an imaginary session (what the tester says, what the user says).  This book is for light reading.

Video: Demo of a simple website usability test, by Steve Krug

Rocket Surgery Made Easy by Steve Krug: Usability Demo
This 24-minute video demonstrates a simple website usability test conducted by Usability Consultant, Steve Krug.  This is a great demo of how simple a usability test can be.

Jana Brech

About Jana Brech

Hi, my name is Jana Brech. I live in Brisbane, Australia. After 20+ years in the IT industry as an Analyst/Programmer, I have a passion for Usability. I love good design. Drop me a note to say hi, or leave a comment. Web Consulting, Expert Reviews

Comments

  1. Obaid  October 12, 2013

    Hi
    Short but good .
    New to testing but exp in development and unix.

    Thanks
    obad

    reply
  2. Steve Stephens  February 15, 2016

    Thanks Jana.
    So many wise statements which are completely ignored by some embarrassingly incompetent big businesses: ie Telstra’s home pages.
    Lots of good information, presented well.
    Thanks,
    Steve.
    Queensland Sporting Hovercraft Club newsletter editor.

    reply
    • Jana Brech
      Jana Brech  February 15, 2016

      Thanks, Steve. I hope you find the information helpful.

      I agree with your comments about Telstra’s website. I’d be interested to hear your specific reasons. I find it frustratingly difficult to find the simplest of information on their website. Such as how to turn on and off diversion to their 101 Message bank. What’s your experience of their website like?

      Of course, it’s not just Telstra. So many major players around the world fail to attend to the most obvious usability issues. Much to the frustration of everyday users. I could say a lot about that.

      But right now, I’m interested to hear about the quality of your experience as a user of such corporate websites.

      reply
      • Steve  February 15, 2016

        In a word, hopelessly woefully pathetic usability. Frustratingly slow to load the 100 odd “bells & whistles” & annoyingly irrelevant snippets of just about everything except Telstra’s original core products.
        It doesn’t say much for their usability when you choose to wait on the phone-queue for the next 1/2 hour or more, while amusing yourself by sifting through all their home-page links in the vain hope of accidentally tripping across the required information before eventually being greeted by yet another overseas assistant & scripts.
        (It’s so refreshing to occasionally vent one’s spleen over the ineptitude which affects most Australian adults on a regular basis!)
        You could actively canvas such corporations with a compelling case for your services, but the accountants in charge, & the many sub-managers (who must think they are), are hidden within the many turns from the entry-point of their corporate maze. They often have trouble communicating within their own organisation, so the poor customer has almost no chance.
        You should convince them that they need a VERY basic home-page portal which only includes links to the home-pages for each of their core services: ie Residential, Mobile, Wireless Data, B2B, Home Entertainment, etc. All those home-pages need to continue the minimalist style with concise index links, not popups, icons, clips, etc.
        Good luck with that!

        reply
        • Jana Brech
          Jana Brech  February 15, 2016

          Ah, now I understand what you meant by home pages (plural). Everything you say is spot on. I wouldn’t dream of approaching Telstra to do a usability review, for exactly the reasons you’ve stated. Sadly, I think they lost the plot as a corporation years ago. Their vision and once-held standards of excellence have been sacrificed to the bottom line. As you astutely said, it’s the accountants who are now in charge. In an ideal world, Telstra would have a great service ethic, and I would include your suggestions in a usability review. Thanks for the great input.

          reply

Add a Comment


7 + eighteen =