In single blind paper submissions, reviewers can see the author names and affiliations. Suppose two papers are submitted in a conference, one of them is fairly well written and the other one is a low quality paper. Also, one or two authors are common in both the papers. If both the papers go to same reviewers for review and reviewers review the low quality paper first, do they become biased while reviewing the fairly well written paper? Do the chances of selection of the good paper become less?
Do reviewers become biased about research papers written by authors of a previous low quality paper?
7
-
2Double blinded process is designed to avoid a bias, so the answer is quite probably yes.– Ander BiguriCommented Mar 3, 2016 at 18:49
-
If the papers only share one or two authors but are otherwise different (i.e., not two papers about almost the same thing), it's very likely that they will get reviewed by different people, making the whole issue redundant.– PeterisCommented Mar 3, 2016 at 19:09
-
@Peteris Not true when the conference has a bidding system, as shared authors --> similar topics --> likely overlap in set of matched reviewers.– jakebealCommented Mar 3, 2016 at 19:29
-
@Peteris: I'm not very sure, but in many computer science conferences, authors have to select a suitable topic/track. Papers with common authors makes it more likely for papers to be of same topic and hence reviewed by same reviewers.– kunal18Commented Mar 3, 2016 at 19:34
-
You might want to clarify what you mean by "low quality" paper. The research is poorly done, or it is good research, but poorly written up. These two things give very different impressions.– KimballCommented Mar 4, 2016 at 1:25
|
Show 2 more comments
2 Answers
I would distinguish between two papers in parallel (as in your example) versus a history of bad papers.
If I'm seeing a good paper and a bad paper that share an author, and I don't yet know that author well, then the bad paper probably won't affect my opinion of the good paper too badly (unless we are talking really embarrassingly bad). I'll have a mixed opinion, but I'm still forming it.
On the other hand, there are certainly some authors who have built up a history with me such that I see their paper and think: "Oh no, I hope they've learned from last time."
-
Thanks! Although in the question I've mentioned that few authors are common, but what if none of the authors are common but affiliations are same? Does it matter or influence you?– kunal18Commented Mar 3, 2016 at 17:47
-
9@stalin Shared affiliations only? Not even a little bit of influence: every academic institution is a heterogeneous mess, and the traditions of independence mean I'll expect little correlation between two disjoint sets of authors.– jakebealCommented Mar 3, 2016 at 17:49
-
4@FábioDias That's a nice principle, but absolutely not true in practice. Consider encountering an under-specified methods description: if I know that the authors usually have good methods, I'm more likely to give them the benefit of the doubt and ask them to clarify their methods on revision; if I know the authors usually have sloppy methods, I'm likely to assume that poor specification implies poor execution.– jakebealCommented Mar 3, 2016 at 17:53
-
2@jakebeal Knowing this bias, wouldn't you fight against it? Wouldn't that nullify (hopefully) the bias? For instance, I hate badly written articles, but I try to not let that influence the other aspects of the work. Personally, I try to review, and live, by that principle... I won't promise that I'm, but, contrarily to yoda's beliefs, there is try :) Commented Mar 3, 2016 at 19:17
-
3@FábioDias As I said, it's a good principle to hold. I simply know that, being human, I am not able to hold to it in practice. That's an important reason that some organizations go for double-blind review.– jakebealCommented Mar 3, 2016 at 19:25
In psychology, there is the phenomenon of "anchoring" - a first impression colours the rest of a transaction, whether it is negotiating a price, opinion about personality or other issues.
This indicates that, yes, a bad first paper which captures the attention of the reviewer can have a detrimental effect on how subsequent work is seen. If it is a novel researcher, the memory of the name may not be retained, so, the effect might be milder, but if there are two papers in one conference from the same author, and reviewed by the same people, that effect may indeed hold.
This is a reason why many conferences espouse double-blind reviews (although I am not a friend of this for other reasons which are outside the scope of this question).
answered Mar 3, 2016 at 17:07
-
2I agree that your impression of a researcher colors your first impression of one of their papers. I try to review papers I get critically enough that it mostly overcomes my "first impression biases," but unfortunately many people are not careful thorough reviewers.– KimballCommented Mar 4, 2016 at 1:28
-
1@Kimball Yes, it definitely shouldn't happen. I also take every paper at face value; my psychological trick is that I know that sometimes papers with the same authors have a different lead author, and this shows - I find I cannot always impose my view of style and clarity on my fellow authors... This makes it easier for me to treat the paper as what it presents itself as, rather than what I expect it to be. Commented Mar 4, 2016 at 9:44