74

Yes, there is one and only one standard method that is universally employed by reputable academic institutions worldwide. This is how you evaluate a researcher: Read their papers. Attend one of their talks. Ask the opinion of other experts in the field. This is how hiring committees and promotion committees do their job. There are no shortcuts. Parts 1-...


59

As much as I like Google Scholar, requiring candidates to create a Google Scholar profile specifically seems inappropriate. You are effectively saying you won't hire people that don't use Google. What you could do is make it an optional part of the application or you could ask candidates to submit something more vague like a "citation report" and suggest ...


45

I feel there are two different layers to that - whether, and to what extent, teaching evaluations are actually anonymous, and if they aren't, whether it's still "safe" to give a bad one. Are evaluations anonymous? On a superficial level, all universities that I have taught at had entirely anonymous evaluations. At no point in the process was I ever told ...


44

Have you considered ORCID (Open Researcher and Contributor IDentifier)? I have the same concerns about intellectual property protection issues around Google unfortunately. GoogleScholar is also quite discipline specific (as others have said here) and is banned in some countries (China, etc). So to endorse a product that exposes a scholar to legal ...


27

There is no straightforward answer to this, since it varies enormously by institution. Anonymous feedback can still be unmasked As others have said already, an ostensibly anonymous survey may not end up being so anonymous in practice; it is pretty easy to guess someone's identity, especially if the class is small or if you have interacted with the lecturer ...


26

If one claimed that a particular scholar was "above average" or "noted" in their field, is there any good metric by which to support or deny such a claim? No. As a rule of thumb, this isn't the kind of thing that you can measure with a metric. Elvis Presley was the king of rock and roll. Why? Is it because he pumped out more albums than the others? Because ...


16

I do not think it is reasonable to ask candidates to create a profile on any third-party platform. Particularly on a google service, taking in account that some proportion of web users have concerns about this company (as well as other large corporate data processing companies), and do not want to get on their radar if possible. Typically, it is sufficient ...


16

Scientific papers are usually published in journals and conference proceedings. Publishers report at least the publication year and month, but may also report more information, like the submission date, the revision date and the date of the first online publication. Your example is just a preprint, not the publisher's version, and does not contain any ...


12

It is common, and reasonable, for employers to require job candidates and employees coming up for review to provide the employer with any information it needs to evaluate the candidates/employees. So certainly you can ask them to prepare readable, well-formatted publication lists, citation information, and anything else that lets you evaluate their impact ...


11

While I think your "justice" metaphor is wrong, the answer in all cases is experience. You need experience in the field as a reviewer. You need to know what has been done and what is important yet to do. As an editor you need either that field experience or experience with your reviewers - who is trustworthy and who is not. As an author in the early career ...


10

If you want to rank two professors against each other, you might be tempted to use the h-index. Don't. As many of the other answers point out, it's a severely flawed metric, and it doesn't really tell you a lot. However, if you want to figure out whether a given professor can reasonably be described as "noted" or "outstanding", then that is a quite ...


8

Is there a generally accepted way to indicate that a particular professor or scholar is outstanding, or above average, in their field? The generally accepted method for assessing a particular scholar's merit is to familiarize oneself with their work. Such an assessment requires a solid basis of expert knowledge. I understand there are certain indicators,...


7

I can't say exactly how common it is, but it happens. But it is most likely in the first couple of years of a probationary (tenure track assistant professor) contract and less likely for tenured professors. It might even be required each once in each year of the contract. And if problems with teaching have arisen, then it is more likely that it will be ...


7

Don't sanction; just ask the lecturer to complete their self-assessment and re-compute their overall teaching evaluation. In the future, consider introducing a process that prevents lecturers forgetting to complete the self-assessment, e.g., by asking for the assessment more than once.


6

I think you have a misconception here about the point of the peer-review. The review process is not meant to be a future prediction of how many citations a paper might get. Yet, I have experienced in the past that some journals ask the reviewers, after the review is finished, if a paper should be highlighted in the current issue of the journal or on the ...


6

Assuming your university has a subscription, Scopus is pretty good at giving you relatively comprehensive and up to date author publication and citation profiles. It's generally good at dealing with name conflicts. A few scenarios where it might fail: Academics who have changed names (e.g., by marriage). Academics with particularly common names who have ...


5

The particular paper you linked is a conference paper. Its date of publication will be the date of the conference, which in this case was ESORICS 2012, held September 10-12, 2012. One suspects that what you have linked is a preprint - that is the form of the paper submitted to the publisher before it is published. Computer science is often communicated ...


4

My question is would it be a reasonable requirement to force the candidates to create a Google Scholar account You should recommend that candidates provide a Google Scholar Profile (not account). In practice, hiring committees are going to go look for a profile. You might as well let candidates know that is going to happen. You cannot force job ...


3

It is hard to give you well-targeted advice, not knowing a lot more, but I can offer a few suggestions for thinking about the issues. First, your advisor's past comments may just have been momentary frustration that should probably not have been expressed. Perhaps he just has a volatile personality. But you seem to have convinced him now, so no problems. ...


3

I have seen my fair share of mathematicians failing basic tasks. Probably more iconic ones are related to arithmetics but that is more understandable. For example I have seen a +40 years old working mathematician volunteering to teach differential equations because s/he didn't feel confident in his/her ability do solve them and wanted to force him/her-self ...


3

It is very unlikely that you will suffer negative consequences from it and there are no real avenues to retaliate against you. At best, the wronged instructor may try to do something like spread rumors against you, but honestly, with how busy everyone is, do you really think anyone has the time? Extremely petty and irrational people may try to retaliate ...


3

I will only supplement the great answer of xLetix here. Let me note that the more radical and extreme your statements in an evaluation the less likely it will be that it is acted upon. If you have serious, negative, things to say, then another venue will probably be more effective, though it is unlikely to be anonymous. I think that extreme comments are ...


3

The negative proof to the question here is far broader than academics: is there a metric for the best car? Best parent? Best programming language? Smartest person? No, because all these things have many orthogonal dimensions that simply can't be collapsed to one without unacceptable information loss. Researchers can be creative, well funded, methodical, hard ...


2

I disagree with the top answer; it is not evidence-based, as in fact the evidence shows that penalizing wrong answers harms students who are underconfident, and in particular this statistically affects women more often than men. Usually the system goes like this: Good answer = get some points, no answer = no points, wrong answer = negative points. ......


2

All metrics that used (e.g. number of first/senior authorships, sum of impact factors, percentile ranks of impact factors, citations, H-index, grants and other funding etc) have all their advantages and many more disadvantages. Never the less they are used in hiring processes in one or the other way because otherwise it is not possible to assess several ...


2

To some extent this depends on the field and where in the process you are (although I have to find it funny that 30 applications is considered severe; in math there are positions which get literally 300 or 400 applications). Here are some relevant considerations: Are you in a field which usually uses Google Scholar? Math for example doesn't almost at all(...


2

Yes, it is completely within reason to ask candidates to have a profile somewhere. What it's not appropriate is to demand they use Google specifically. Besides Scholar there are other alternatives like Scopus, ORCID (mentioned in other answers) or even creating profiles in sites like scholarly, researchgate, or academia.edu. You should leave it to them to ...


1

Your story reminded me of the Monty Hall problem. Even great mathematicians didn't believe: Many readers of vos Savant's column refused to believe switching is beneficial despite her explanation. After the problem appeared in Parade, approximately 10,000 readers, including nearly 1,000 with PhDs, wrote to the magazine, most of them claiming vos Savant was ...


1

Is it reasonable to ask candidates to create a profile on Google Scholar? Absolutely not. Google is an atrocious entity involved in mass commercial and governmental surveillance, political censorship etc. You really must not require people to use Google's services, legitimizing these practices. Now, to be practical - I'm not saying that you should demand ...


1

I've been teaching mathematics for +25 at all levels from high school to PhD. Feedbacks are a good thing, but useful feedbacks are rare. As to the original question -- I am sorry to read that there are fears of retaliation. This obviously depends of your environment. Meet the teacher in person, or a colleague of his. If not possible, send him an email. If ...


Only top voted, non community-wiki answers of a minimum length are eligible