My team is developing a tool which solves certain problems we encountered in our work. We decided to describe the tool in a paper and attempt to publish it. We got stuck on evaluating the usability of our tool.

We found a SUS measure - Subjective Usability Scale, which appears to fit our case. It consists of a few questions that users of the tool should answer. The problem here is, that our tool has only been used by our team (sample size 7) and it is difficult to get a broader range of users to test it.

In such case, do you think we should include the evaluation using such scale based on our team members only? Or should we rather skip the evaluation part completly and simply describe the tool leaving the evaluation to the reader based on the features of the system we described. If we should include it, what statistical method should we use to compare our results with the reference results of the SUS scale?


  • 3
    This is a question about research methodology, usually not considered on topic for this site. It varies a lot by subfield what is considered an adequate user study; in my field I think it would be pretty frowned upon to evaluate a tool using a survey of the team who created it. Commented May 8, 2020 at 14:08

1 Answer 1


We decided to describe the tool in a paper and attempt to publish it.

Why do you want to publish the description? Do you expect that other people will want to use this exact tool? Then you may need somebody outside the original team to evaluate it. Do you expect that nobody will use this exact tool, but the ideas and concepts might be useful to others in developing their own tools? Then the usability aspect is probably irrelevant.

Not the answer you're looking for? Browse other questions tagged .