I was asked for the first time to be a reviewer for a journal paper.

If it was a normal paper (proposing a new method, proofing something etc.) I would know how to review it roughly. However, it is a paper about a open source software library and I don't know what is expected of me to review here.

They mostly describe what the library can and cannot do and how it compares to other libraries. Am I supposed to go through the whole code base and test if everything is working, or only if all interesting information are given and nothing is misleading?

Browse other questions tagged or ask your own question.