0

I was wondering if there are any computer science related conferences or journals that review their submissions publicly. So everyone can see what the authors submitted and what the reviewers responded with. I was told that this will create bias between academics but I don't think that will be the case because if someone give a biased review then the community will see that and he will probably hurt his image.

Edit:I have a stack exchange concept on my mind.

Improve this question
5
  • 2
    I've never heard of any such. But why are you interested in this? As you suggest, it may be problematical to try it. In particular, authors might well be reluctant to submit, knowing that their flaws would be immediately public. Blind review lets authors revise work before it becomes very public.
    – Buffy
    Commented Oct 23, 2018 at 11:52
  • It would be good because there would be very difficult for reviewers to be biased or/and lazy. Also if you are submitting a paper for review you would give the best version you already have. I don't see the reason you would be reluctant to do so. On the opposite you could get a lot of constructive feedback about your paper.
    – Anoroah
    Commented Oct 23, 2018 at 14:03
  • 1
    If the submitted paper and reviews will be visible for everyone, what happens if the paper is rejected (possibly based on a biased review)? Is there any risk that it'd be considered a prior publication and make publication elsewhere more difficult? (E.g. an IEEE conference website is at least somewhat different from a regular preprint server...)
    – Anyon
    Commented Oct 23, 2018 at 20:14
  • This seems like a promotion, not a question. If you want to discuss such things, the chatrooms are a better venue. Or maybe Meta. But I doubt that you will convince very many people.
    – Buffy
    Commented Oct 23, 2018 at 20:44
  • I am not trying to convince anyone. I am just asking if there are any open review conferences/journals.
    – Anoroah
    Commented Oct 23, 2018 at 22:55

3 Answers 3

Reset to default
3

You are looking for conferences or journals that have open reviews, that is, the submitted papers are visible to everyone, as well as the reviews. Possibly, the reviewers are even known to the public.

There are a few such conferences and journals. The Semantic Web Journal is a computer science journal where all submissions are visible to everyone and reviews are made public. By default, reviewers are known, unless they opt out and be anonymous (although their reviews become public). Authors of rejected papers can ask to have their submitted papers and accompanying reviews removed from the web site if they want. This journal is quite successful and has created a trend in the area of Semantic Web research to provide open reviews in related conferences. The European Semantic Web Conference 2018 had open reviews.

This is not the only example of open review process. I have seen another journal adhering to the open review process, although I don't remember which one right now.

Studies on the respective merrits of open vs. single blind vs. double blind reviews seem to be inexistent or very limited at the moment. Studies comparing reviewing policies seem to be focusing mostly (as far as I know) on comparing single blind vs. double blind, ignoring the open review scheme. So it is hard to say whether it is beneficial or not. There are people who are strongly against open reviews (citation needed, but I talked to some of them) and there are people strongly in favour of it (citation needed, but I talked to some as well).

I am myself a supporter of open reviews. I reviewed papers openly. You can see my open reviews for the Semantic Web Journal there: review on ActiveRaUL: Automatically Generated Web Interfaces for Creating RDF Data, review on Semantic Web Machine Reading with FRED (first submission), review on Semantic Web Machine Reading with FRED (revised version), review on Semantic Web Machine Reading with FRED (last version), review on loomp - mashup authoring and semantic annotation using linked data, review on Survey on complex ontology matching. I would disclose all of my reviews if I was allowed to do so.

I made other open reviews that have been removed because the authors requested it.

Improve this answer
1

Fair warning: I'm going to pretend that the question is: why do all the conference/journals use blind peer review?

The rationale for blind peer review is simple: if the authors knew the reviewers, they could try to maximize their chances to get published through many unethical means (bribing, threatening, ...). Even without going that far, a reviewer would be tempted to curb their criticism with an author they know, especially if they hope to get a job at their institution or collaborate with them (btw this is why the standard is more and more often double blind review: the reviewer doesn't know the author either).

This is why blind review is the standard: while not perfect, it ensures a certain level of honesty and objectivity in the review process, and in turn a certain level of trust in the accepted papers.

Improve this answer
4
  • You got some really good points but your answer is quite far from my question. But still I can not agree with you. The submitted paper and the reviews will be public for everyone to check. If an author "persuaded" a reviewer to approve his paper, that will be bad for the reviewer because he gave a good review to a bad paper and his reputation will be affected. I think I should point it out on my question but I have a stack exchange concept on my mind.
    – Anoroah
    Commented Oct 23, 2018 at 19:13
  • I guess this means transferring the responsibility from the editor to each individual reader, so thousands of person doing the same job instead of one. For example if somebody wants to study the state of the art in some specific domain, they would have not only to read the papers, but also check that they can trust each paper by reading the reviews and potentially the reviews of the reviews, and why not even at the third level... And we would likely see parasite companies paying people for writing fake reviews and all sorts of similar problems.
    – Erwan
    Commented Oct 24, 2018 at 18:51
  • Or we could just have the same system but with open reviews. And correct me if I am wrong but don't these things happen in blind reviews as well?
    – Anoroah
    Commented Oct 24, 2018 at 20:08
  • Imho the real problem with reviews is that people are not paid to do them. That would give reviewers a sense of responsibility and a good reason to do the job carefully. But the academic world kind of pretends to be free from economic constraints, so it's not an option it would seriously consider. Anyway this is only my opinion and it's quite far from the original topic.
    – Erwan
    Commented Oct 26, 2018 at 10:32
1

https://openreview.net/ I wish all the journals/conferences did double blind reviews, it is the -only- ethical way. There are many single blind review systems that makes me cringe. Last time, I had a lot of bias and unjustified assumptions due to this. Making them publicly available is a step further, so you can find venues that do that by following the link.

edit: I read your question wrong. Why are you interested in non blind reviews? Well, I think on openreview, if you pull your submission out then the name of authors, and reviewers (not sure) will show up. I think this is to prevent future accusations of stealing ideas.

Improve this answer
4
  • In double blind reviews one can argue that it is not impossible to have an idea about who submitted the paper (specific field, citations, maybe writing style). I am looking for open review conferences / journals because on my point of view seem the most ethical/right way to publish a paper.
    – Anoroah
    Commented Oct 23, 2018 at 22:57
  • It is true that you can "guess" who did it, famous labs are known to cite their previous work a lot, but when names and affiliations are known, either it will be another lab in competition and they will mercilessly criticize your work, I mean it is okay if they were being objective but last time I had this reviewer's comments they were flowing with subjective bias, or it will be a positive bias because they associate your name or affiliation and think oh it must be good. There are many pals looking out for and favoring each other's lab's works as well even if they are not great papers.
    – dusa
    Commented Oct 24, 2018 at 7:05
  • What is your topic? Computer science journals/conferences is just too broad. Why don't you check "call for papers" for conferences or "author" section of journals of a couple? This information is available for each conference/journal.
    – dusa
    Commented Oct 24, 2018 at 7:10
  • 1
    That is another reason to submit a paper to a public review conference/journal if the reviewers will be biased everybody will see it. I have submitted some papers and a lot of the reviewers made some really wrong comments. If those reviews were public they would really embarrass their selves.
    – Anoroah
    Commented Oct 24, 2018 at 10:35

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .