10

In Brazil, a government agency, named CAPES, takes care of evaluation of the postgraduate courses. Among other things, they receive criticism, because such evaluation is based on bibliometric approach. Explaining briefly, the main problem of following this approach is that it becomes counterproductive, because the academy spends time on increasing the number of publications, instead of focusing on quality research (it is considered more valuable to increment the postgraduate course classification).

A logical advice would be that the organization needs switch to a qualitative approach (here is a report, where four international guests shared this observation). My questions are the following.

Are there today any countries that can be models in regard to their system of the academic production evaluation? How such transition can be initiated, focusing on a qualitative approach?

Improve this question
  • 1
    I think you mean "productivity". – David Ketcheson Sep 20 '15 at 6:12
  • In the UK, you are only evaluated based on a small number of publications every 5 years. Additional publications in that time span are not considered. – David Ketcheson Sep 20 '15 at 6:13
  • 3
    Every time you set up a "measure" (really, some measurable quantity that you hope correlates with some nebulous, often multi-dimensional "quality" that is tenuously related) you set yourself up to be gamed. – vonbrand Sep 20 '15 at 20:42
  • That is a very sensible point. I heard about UK. That also brings some questions: How to guarantee an equal and fair evaluation for everyone? Who will evaluate the evaluators? However, I think that is a much more interesting way to evaluate a researcher. I would like to see more studies addressing this evaluation perspective. Thanks for the answers! – rwehresmann Sep 21 '15 at 11:18
1

I'm from Brazil and I understand you very well about our research/funding system.

Let's branch the problem in topics:

  • CAPES classify the papers by "research impact", it's basically how hard is the congress/conference... to accept your paper: (A1,A2,B1,B2,B3,B4,B5,C).

  • CAPES is responsible for funding research presentations and they have a very limited money.

  • Only A1, A2 and sometimes B1 are really relevant for the society. Nevertheless both are commonly international congress. (let's consider international travels are expensive in Brazil, plus the conference taxes)

  • Let's also consider, poor relevance conferences (B2,B3,B4,B5,C) are commonly in Brazil, sometimes on places where there are a lot of universities, e.g São Paulo, Rio, Rio Grande do Sul, etc. Furthermore, all of them are in Portuguese.

Well, at this time I guess you already see the problem... Summarizing, the Brazilian research is very correlated with local congress and that makes everything easy even for CAPES (due to funding), even for researchers (due to easy paper acceptance), then you create a perfect productivity system... With a lot of useless research.

In my opinion, poor impact research are irrelevant. Then, I guess the solution would be change the impact rank and funding to only classify (A1, A2 and B1). This probably would make everyone work hard and then you probably would change the status quo in a long-term.

  • Note that is theoretical scenario. As I said before, this kind of system is perfect for lazy researchers and their financiers.
Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.