Apart from a SUS (System Usability Scale) assessment – which is an established method of measuring usability – there are several other assessments that can be used for a website.
The SUS assessment is a good method for measuring the users experience of a website and how they grade the usability of a system, but it does not give any indications to what can be improved or where the users get stuck. A quality assessment can give you more values. Now, we’re still talking about experiences of a website, and not specific tasks or paths a user will take though a website, we are measuring the feeling associated with the system. Before you start you should also have a good reason for measuring and a plan for how to treat or act on the result.
How to do a quality assessment
A quality assessment can be done both with user testing and though a questionnaire on a website. The advantage of doing it face-to-face in user tests is that you can dig deeper and discuss problems and solutions with the user. The advantage of using a questionnaire is that it will be faster and you can do the assessment on a quantitative level, getting statistically reliable results.
When conducting this type of assessment in user testing note that you will generally get a higher score than if you conduct the same assessment (of the same website with the same type of users) in a questionnaire. The reasons for this could be that:
- users that are willing to come in to do a user test can be generally more inclined to like the company or product.
- it is harder for users to criticise or give a lower score when face-to-face with an interviewer.
I would therefore recommend doing quality assessments of a website in an anonymous questionnaire. The important thing is that you do the same thing each time you assess, and not use face-to-face interviews in the first assessment, and a questionnaire in the follow up and so on, if you do, you would be comparing apples and pears.
The purpose of a quality assessment
The purpose of a quality assessment is to measure specific experience and content that is influential of a website. How well the user experiences and values these qualities. The assessment can contain questions about navigation, how relevant the content is, brand values, and comparisons with competitors.
Formatting the questions
Formulate the questions positively. It will make it easier for the user not to have to re-think the question and order of answers each time. For example say: “I think the website is very usable” instead of: “I think you could improve this website”. Try to choose the most relevant questions you want answers to and keep the questions limited – preferably not more than 20.
Examples of questions that can be used to assess different experiences
- The menus on the website are easy to understand.
- I find what I am looking for in the menus.
- I think it’s easy to search and find content/articles/information I am looking for.
- The website helps me develop my skills in… (the subject the website is about).
- The website helps my personal development in… (the subject the website is about).
- The website helps me get on top of… (the subject the website is about).
- It’s easy to learn about the products on the website.
- It’s easy to… (do which ever task the website is used for, for example buy a book).
- It’s easy to find something to buy on the website.
- It’s easy to find my ongoing… (purchases, discussions or other).
- It’s easy to find historical… (purchases, discussion threads or other).
- It’s easy to find customer service.
- It’s easy to get help.
- The website is fast.
- The website helps me get new ideas.
- The website inspires me to change.
- The website raises a lot of thoughts about my lifestyle.
- The website is… modern (or whichever words are connected with the brand).
- The website feels secure to use.
- The content of the website is relevant to me.
- The products on the website are relevant to me.
- I would recommend this website to others.
- It is easy to personalize the website and create my own profile.
- It is easy to see which content is mine and which belongs to others (in for example a discussion forum).
Comparing to competitors
- In relation to other… (fashion websites, blogs, news sites) ”Website name” is better.
- In relation to other… “Website name” has a better range of products.
- In relation to other… “Website name” has better service.
The scores will vary between 1 and 5. 1 means the users gives the website a bad grade or think it’s is difficult. 5 is easy. 3 is average.
The replies the users can choose from will be:
1 point: I don’t agree at all.
2 points: Doubtful.
3 points: Average.
4 points: I agree somewhat.
5 points: I totally agree.
When to measure
You first need to do a baseline assessment on the existing website – before any changes or improvements have been done. After introducing improved functionality on the website some time should lapse before measuring again, depending on how frequently the user visits the website. If it’s a website the user visits every day the assessment can take place a week or two after the improvement has been done – to give the user a little bit of time to get used to the new functionality. If the user is less frequent it is better to wait a few weeks before measuring.
Which users should be used?
If the company has different kind of user groups, target groups or personas it is of course very interesting to see if the answers vary between the different groups. If you have not divvied up your users into different groups, you could do so based on how frequently they log in, how often they use the product or how often they buy something, so you don’t end up spending the most time fixing stuff for the least interested.
Analyze the results
Once you got enough answers you can read the results in many different ways and work out a score for each question. Which function may the user benefit most from improving? What is wise to continue doing? What should you stop doing? Everything of course depends on the reason you set for assessing the website.