SUS stands for System Usability Scale. SUS was developed by Brooke (1996) as ”the quick and dirty” assessment tool – it was used to quickly and cheaply assess a product or a service. (1). A low cost usability test.
A SUS test is supposed to cover:
- Effectiveness (the ability of users to complete tasks).
- Efficiency (how difficult it was to perform the task).
- Satisfaction (how the users felt when performing the task).
A system can be a car, a website or a product. SUS assessment has a reliability value of 0.85 (2).
A SUS assessment is constructed of 10 statements that are valued on a 5 point scale, according to how much the user agrees or disagrees with the statement. The result can vary between 0 and 100, where a higher score indicates better usability. Every other question is a positive statement and every other question is a negative statement, so both the user and the person analyzing the result have to stay focused!
What counts as a good result?
Different publications mention different averages for the assessments.
In An Empirical Evaluation of the System Usability Scale (3) the author has done 2324 assessments with an average on 70.14 based on all the assessments. When the assessments are divided up into each project the average is 69.69. The same author claims that good systems get between 70-80 points and exceptional systems get 90 or more.
Jeff Sauro writes in Measuring Usability with the System Usability Scale (SUS)(4) that he – from 500 assessments – has arrived at an average of 68. He also states that everything over 68 is therefore over average. A value over 74 is the grade B and a value over 80.3 is grade A. Points below 51 will result in an F (fail).
How many users should be tested?
The opinions of how many users should be tested vary as well. In An Empirical Evaluation of the System Usability Scale(2) the author claims that for internet-based web pages 50 users are needed to get a good result.
In Measuring Usability with the System Usability Scale (SUS)(4) the author states that the user group and the reliability of the test is unrelated and a SUS assessment can be done with as little as two users and still generate reliable results.
This form is taken from SUS – A quick and dirty usability scale (1)
Analysing and calculating the results
Every other question is a positive statement; they are the uneven numbered questions 1, 2, 3, 5, 7 and 9. The user will fill out a value between 1 and 5. From that value 1 point is deducted. If the users rate the statement as a 4, you subtract one (4-1) and arrive at the result of 3.
Every other statement is a negative statement; they are the even numbered questions 2, 4, 6, 8 and 10. The user will enter a value between 1 and 5. This value is then deducted from the number 5. If the user gives the statement a 4, you will subtract 4 from 5 (5-4) and arrive at the result of 1.
Yes, a little complicated, but once you get going it will become easier! See the example below and things may become a little clearer. The positive questions have a green background and the negative questions a red background.
Every question will get a value of between 0 and 4. They are then added together and multiplied by 2.5. The final result will end up somewhere between 0 and 100.
- Brooke, John. SUS – A quick and dirty usability scale. Fetched from http://hell.meiert.org/core/pdf/sus.pdf
- Kirakowski, J. (1994) The use of questionnaire methods for usability assessment. Unpublished. Fetched from http://sumi.ucc.ie/sumipapp.html
- Aaron Bangor, Philip T. Kortum & James T. Miller (2008): An Empirical Evaluation of the System Usability Scale, International Journal of Human-Computer Interaction, 24:6, 574-594- Fetched from http://www.tandfonline.com/doi/abs/10.1080/10447310802205776
- Sauro, Jeff Measuring Usability with the System Usability Scale (SUS). Fetched from http://www.measuringusability.com/sus.php