74

Yes, there is one and only one standard method that is universally employed by reputable academic institutions worldwide. This is how you evaluate a researcher: Read their papers. Attend one of their talks. Ask the opinion of other experts in the field. This is how hiring committees and promotion committees do their job. There are no shortcuts. Parts 1-...


59

As much as I like Google Scholar, requiring candidates to create a Google Scholar profile specifically seems inappropriate. You are effectively saying you won't hire people that don't use Google. What you could do is make it an optional part of the application or you could ask candidates to submit something more vague like a "citation report" and suggest ...


45

I feel there are two different layers to that - whether, and to what extent, teaching evaluations are actually anonymous, and if they aren't, whether it's still "safe" to give a bad one. Are evaluations anonymous? On a superficial level, all universities that I have taught at had entirely anonymous evaluations. At no point in the process was I ever told ...


44

Have you considered ORCID (Open Researcher and Contributor IDentifier)? I have the same concerns about intellectual property protection issues around Google unfortunately. GoogleScholar is also quite discipline specific (as others have said here) and is banned in some countries (China, etc). So to endorse a product that exposes a scholar to legal ...


27

There is no straightforward answer to this, since it varies enormously by institution. Anonymous feedback can still be unmasked As others have said already, an ostensibly anonymous survey may not end up being so anonymous in practice; it is pretty easy to guess someone's identity, especially if the class is small or if you have interacted with the lecturer ...


26

If one claimed that a particular scholar was "above average" or "noted" in their field, is there any good metric by which to support or deny such a claim? No. As a rule of thumb, this isn't the kind of thing that you can measure with a metric. Elvis Presley was the king of rock and roll. Why? Is it because he pumped out more albums than the others? Because ...


16

I do not think it is reasonable to ask candidates to create a profile on any third-party platform. Particularly on a google service, taking in account that some proportion of web users have concerns about this company (as well as other large corporate data processing companies), and do not want to get on their radar if possible. Typically, it is sufficient ...


16

Scientific papers are usually published in journals and conference proceedings. Publishers report at least the publication year and month, but may also report more information, like the submission date, the revision date and the date of the first online publication. Your example is just a preprint, not the publisher's version, and does not contain any ...


12

It is common, and reasonable, for employers to require job candidates and employees coming up for review to provide the employer with any information it needs to evaluate the candidates/employees. So certainly you can ask them to prepare readable, well-formatted publication lists, citation information, and anything else that lets you evaluate their impact ...


11

While I think your "justice" metaphor is wrong, the answer in all cases is experience. You need experience in the field as a reviewer. You need to know what has been done and what is important yet to do. As an editor you need either that field experience or experience with your reviewers - who is trustworthy and who is not. As an author in the early career ...


10

If you want to rank two professors against each other, you might be tempted to use the h-index. Don't. As many of the other answers point out, it's a severely flawed metric, and it doesn't really tell you a lot. However, if you want to figure out whether a given professor can reasonably be described as "noted" or "outstanding", then that is a quite ...


8

Is there a generally accepted way to indicate that a particular professor or scholar is outstanding, or above average, in their field? The generally accepted method for assessing a particular scholar's merit is to familiarize oneself with their work. Such an assessment requires a solid basis of expert knowledge. I understand there are certain indicators,...


7

I can't say exactly how common it is, but it happens. But it is most likely in the first couple of years of a probationary (tenure track assistant professor) contract and less likely for tenured professors. It might even be required each once in each year of the contract. And if problems with teaching have arisen, then it is more likely that it will be ...


7

Don't sanction; just ask the lecturer to complete their self-assessment and re-compute their overall teaching evaluation. In the future, consider introducing a process that prevents lecturers forgetting to complete the self-assessment, e.g., by asking for the assessment more than once.


6

I think you have a misconception here about the point of the peer-review. The review process is not meant to be a future prediction of how many citations a paper might get. Yet, I have experienced in the past that some journals ask the reviewers, after the review is finished, if a paper should be highlighted in the current issue of the journal or on the ...


6

Assuming your university has a subscription, Scopus is pretty good at giving you relatively comprehensive and up to date author publication and citation profiles. It's generally good at dealing with name conflicts. A few scenarios where it might fail: Academics who have changed names (e.g., by marriage). Academics with particularly common names who have ...


5

The particular paper you linked is a conference paper. Its date of publication will be the date of the conference, which in this case was ESORICS 2012, held September 10-12, 2012. One suspects that what you have linked is a preprint - that is the form of the paper submitted to the publisher before it is published. Computer science is often communicated ...


4

My question is would it be a reasonable requirement to force the candidates to create a Google Scholar account You should recommend that candidates provide a Google Scholar Profile (not account). In practice, hiring committees are going to go look for a profile. You might as well let candidates know that is going to happen. You cannot force job ...


3

It is very unlikely that you will suffer negative consequences from it and there are no real avenues to retaliate against you. At best, the wronged instructor may try to do something like spread rumors against you, but honestly, with how busy everyone is, do you really think anyone has the time? Extremely petty and irrational people may try to retaliate ...


3

I will only supplement the great answer of xLetix here. Let me note that the more radical and extreme your statements in an evaluation the less likely it will be that it is acted upon. If you have serious, negative, things to say, then another venue will probably be more effective, though it is unlikely to be anonymous. I think that extreme comments are ...


3

The negative proof to the question here is far broader than academics: is there a metric for the best car? Best parent? Best programming language? Smartest person? No, because all these things have many orthogonal dimensions that simply can't be collapsed to one without unacceptable information loss. Researchers can be creative, well funded, methodical, hard ...


3

I would ask your professor, other professors, or people you know in the administration. Presumably, they know how these evaluations are done in your University. My experience, in a research institute in Germany, is that faculty is evaluated on both output and grants obtained. Similarly, the institute as a whole is evaluated on these points. Typically, ...


3

I am employed at one of the German Fraunhofer institutes as PhD student. We are doing a lot of research and are partially publicly funded, but we are not a university. However, my answer might still provide some insight. My research output is precisely measured and evaluated. Each time one of my papers is accepted I need to add it to an internal database. ...


2

From your question alone, it is hard to tell what the research database is being used for. However, at German universities any decision to implement such an instrument will be based on a Beschluss (committee decision) or an Ordnung (committee resolution) passed by a relevant committee (e.g. Fakultätsrat, the faculty board). These decisions are in general ...


2

I won't comment on the specific system, but professors, generally, have to justify their existence. In some places annual reviews are used. IMO, the best of these are initiated by the individual who writes a report giving accomplishments for the year (period) past and setting goals for the future. Normally one comments on how earlier goals were met, or not. ...


2

All metrics that used (e.g. number of first/senior authorships, sum of impact factors, percentile ranks of impact factors, citations, H-index, grants and other funding etc) have all their advantages and many more disadvantages. Never the less they are used in hiring processes in one or the other way because otherwise it is not possible to assess several ...


2

To some extent this depends on the field and where in the process you are (although I have to find it funny that 30 applications is considered severe; in math there are positions which get literally 300 or 400 applications). Here are some relevant considerations: Are you in a field which usually uses Google Scholar? Math for example doesn't almost at all(...


2

Yes, it is completely within reason to ask candidates to have a profile somewhere. What it's not appropriate is to demand they use Google specifically. Besides Scholar there are other alternatives like Scopus, ORCID (mentioned in other answers) or even creating profiles in sites like scholarly, researchgate, or academia.edu. You should leave it to them to ...


1

Is it reasonable to ask candidates to create a profile on Google Scholar? Absolutely not. Google is an atrocious entity involved in mass commercial and governmental surveillance, political censorship etc. You really must not require people to use Google's services, legitimizing these practices. Now, to be practical - I'm not saying that you should demand ...


1

I've been teaching mathematics for +25 at all levels from high school to PhD. Feedbacks are a good thing, but useful feedbacks are rare. As to the original question -- I am sorry to read that there are fears of retaliation. This obviously depends of your environment. Meet the teacher in person, or a colleague of his. If not possible, send him an email. If ...


Only top voted, non community-wiki answers of a minimum length are eligible