Definitions of IEEE RAS review "dimensions"

Question

When reviewing for the IEEE Robotics and Automation Society, you are asked to rate the submission on the following qualities:

• Originality of concepts
• Theoretical contribution
• Technological contribution
• Technical soundness
• Thoroughness of results
• Importance of results
• Relevance of applications
• Clarity of presentation
• Relevance to field

I can't find any guide defining these terms. Some of them are obvious, others less so.

What is the difference between "Relevance to field" and "Relevance of applications"?

What is the difference between "Technical soundness" and "Thoroughness of results"? It's hard to imagine one without the other.

I feel that the IEEE should define what they mean by these terms.

Answer 1

These terms all appear to be rather standard fare for aspects of interest in a technical peer review, so I think following any general peer review criteria (such as in this guide) would be a good start to your review.

Regarding the specific distinctions you've mentioned, you might consider trying to define the terms yourself and then reaching out to the editor for clarification (providing your inferred definitions as a starting point). Based on my own experience, I interpret the terms as follows:

• Relevance to field: is the topic of the paper relevant to other researchers in this field? i.e. is the problem addressed by the authors of interest to the readers of this journal (or members of this society, whatever)?
• Relevance of applications: are the applications of the presented work of interest to current practitioners in this field? Consider where the contribution(s) of the paper might be useful in practice, and if those applications are relevant given the state of the field.
• Technical soundness: are the methods and analyses employed in this work appropriate for the problem, mathematically and logically valid, and properly executed?
• Thoroughness or results: have the authors explored or addressed all of the relevant facets of the problem, or has something important been overlooked?