What are characteristics of "high quality" graduate research as viewed by a hiring committee?

Question

When discussing the ability for a faculty candidate to find a job, SE users often say that the quality of graduate research overshadows the prestige of the school at which they received a graduate degree. As viewed by a a hiring committee, what are characteristics of "high quality" research?

Answer 1

Ultimately quality is a judgement call that every reader gets to make.

For example, a paper in mathematics might be considered to be "high quality" to half a dozen people worldwide who happen to be interested in the main result of the paper (or other work). Especially if the question has been open for a while. But other mathematicians may just say "meh" because that result neither interests them specifically, nor advances their own studies.

Reviewers at journals and conferences need to be able to make such judgements and, generally, they do a good job and we respect those judgments when we find published papers. But not everything that is published is of high quality.

But, some of the criteria that reviewers and others will use are as follows.

Is the solution correct? And reproducible?

Is the problem long standing? Or does it break new ground?

Does it open new areas of research?

Is it novel in some sense? Or just an easy derivation from what is already known?

Could I have done this easily? Or does it surprise me in some way?

Is the result widely helpful to either researchers or society as a whole?

Note that all of the above are judgement calls and all imply some scale with, not interesting at one end and very interesting at the other.

But I think the real intent of what you have read, is that it is more important what you do in your research than the general reputation of the place in which you do it. Reputation is based on many things, not all of which are directly relevant to the quality of research. In particular some places that seem quiet generally may be hotbeds of ideas in some niche of some field.

And note that hiring committees may rely on the judgements of others, such as the work being published. They might also rely on just the judgements of writers of letters of recommendations if those writers themselves are in a position both to know and to be honest. But they might also, in some cases, rely on their own judgement. "Well, yes, this is interesting, but not interesting to us at this time."

Answer 2

The UK has an official answer that might be of use. In particular, the general criteria are:

• scientific rigour and excellence, with regard to design, method, execution and analysis
• significant addition to knowledge and to the conceptual framework of the field
• actual significance of the research
• the scale, challenge and logistical difficulty posed by the research
• the logical coherence of argument
• contribution to theory-building
• significance of work to advance knowledge, skills, understanding and scholarship in theory, practice, education, management and/or policy
• applicability and significance to the relevant service users and research users
• potential applicability for policy in, for example, health, healthcare, public health, food security, animal health or welfare.

There are also specific criteria for each quality level for the various panels (which I have merged):

In assessing work as being four star (quality that is world-leading in terms of originality, significance and rigour), sub-panels will expect to see evidence of, or potential for, some of the following types of characteristics:
• agenda-setting
• research that is leading or at the forefront of the research area
• great novelty in developing new thinking, new techniques or novel results
• major influence on a research theme or field
• developing new paradigms or fundamental new concepts for research
• major changes in policy or practice
• major influence on processes, production and management
• major influence on user engagement
• outstandingly novel in developing concepts, paradigms, techniques or outcomes
• a primary or essential point of reference
• a formative influence on the intellectual agenda
• application of exceptionally rigorous research design and techniques of investigation and analysis
• generation of an exceptionally significant data set or research resource.
• a primary or essential point of reference
• of profound influence
• instrumental in developing new thinking, practices, paradigms, policies or audiences
• a major expansion of the range and the depth of research and its application
• outstandingly novel, innovative and/or creative.

In assessing work as being three star (quality that is internationally excellent in terms of originality, significance and rigour but which falls short of the highest standards of excellence), sub-panels will expect to see evidence of, or potential for, some of the following types of characteristics:
• makes important contributions to the field at an international standard
• contributes important knowledge, ideas and techniques which are likely to have a lasting influence, but are not necessarily leading to fundamental new concepts
• significant changes to policies or practices
• significant influence on processes, production and management
• significant influence on user engagement
• novel in developing concepts, paradigms, techniques or outcomes
• an important point of reference
• contributing very important knowledge, ideas and techniques which are likely to have a lasting influence on the intellectual agenda
• application of robust and appropriate research design and techniques of investigation and analysis
• generation of a substantial data set or research resource.
• an important point of reference
• of considerable influence
• a catalyst for, or important contribution to, new thinking, practices, paradigms, policies or audiences
• a significant expansion of the range and the depth of research and its application
• significantly novel or innovative or creative

In assessing work as being two star (quality that is recognised internationally in terms of originality, significance and rigour), sub-panels will expect to see evidence of, or potential for, some of the following types of characteristics:
• provides useful knowledge and influences the field
• involves incremental advances, which might include new knowledge which conforms with existing ideas and paradigms, or model calculations using established techniques or approaches • influence on policy or practice
• influence on processes, production and management
• influence on user engagement.
• providing important knowledge and the application of such knowledge
• contributing to incremental and cumulative advances in knowledge
• thorough and professional application of appropriate research design and techniques of investigation and analysis. • a recognised point of reference
• of some influence
• an incremental and cumulative advance on thinking, practices, paradigms, policies or audiences
• a useful contribution to the range or depth of research and its application.

In assessing work as being one star (quality that is recognised nationally in terms of originality, significance and rigour), sub-panels will expect to see evidence of, or potential for, some of the following types of characteristics:
• useful but unlikely to have more than a minor influence in the field
• minor influence on policy or practice
• minor influence on processes, production and management
• minor influence on user engagement.
• providing useful knowledge, but unlikely to have more than a minor influence
• an identifiable contribution to understanding, but largely framed by existing paradigms or traditions of enquiry
• competent application of appropriate research design and techniques of investigation and analysis.
• an identifiable contribution to understanding without advancing existing paradigms of enquiry or practice
• of minor influence.

Answer 3

It's published in a reputable journal or reputable book press. "Reputable" means it has a reputation for having some quality control, as opposed to a predatory publisher.