17

Computer Science and Engineering have a publication system that is quite different from that of other disciplines, in that conference proceedings have great importance and partially take the role of journal papers (see e.g. this answer or this question).

As an outsider from a neighbouring field, I can see some disadvantages of this system:

  • tight time constraints in authoring and refereeing actively impact the quality of publications: a paper is not submitted or accepted "when it is good enough", but rather when the deadlines force people to act.
  • there is a certain duplication of content between conference and journal papers.
  • travel fees (pre-pandemic, at least) add an unnecessary component to publication costs, and raise the entry bar for researchers from some countries.

On the other hand, I find it harder to pinpoint clear advantages of the system. If you wanted to convince my field to switch to this system, what arguments would you use? What are its scientific benefits? How does it lead to a better research ecosystem?

This system is relatively recent, so one would hope that it solves some issues with the older model.

Feel free to challenge the frame if you disagree with some of the disadvantages stated above --- that would answer my question, too.

10
  • 2
    Why do you say that proceedings are harder to locate than journal papers?
    – GoodDeeds
    Jun 27 at 12:19
  • 5
    @GoodDeeds In my experience articles from old or less-popular conferences are harder to locate than journal papers. It seems that the "tail" of what libraries buy and databases index is shorter for conferences. That might just be my experience; I don't have statistics to back it up quantitatively though. Is your experience different? Jun 27 at 12:47
  • 4
    Just a sidenote, but regarding issue 1, I have the impression that me and most people in CS I work with are "deadline-driven". We don't get anything done unless there's a deadline attached to it. But when that's the case, we absolutely put in all the time and effort required to make it a strong paper, because we want it to be accepted. Jun 27 at 13:18
  • 2
    @FedericoPoloni In CS/Eng where conference proceedings are more important than in other fields, they are not at all difficult to locate, so I would say that point is invalid. Your other three points, however, are all very valid and all make me concerned on a fairly regular basis (as someone who works in the field). Jun 27 at 14:09
  • 1
    @Coder I don't see any clear reason why that's more likely to happen at conferences than journals. There will always be authors that try to get away with the minimum effort, both at conferences and journals. Jun 27 at 17:50
12

I can only really answer specifically for machine learning research, but the purpose of conferences is to avoid the long and inconsistent review times, and the possibly tedious and slow revision cycle, of journal publications. Depending on the journal, receiving the reviewer response can take anywhere from 1-2 months to about a year. And the average paper goes through 1 cycle of revisions, so we can basically double that timeline, and that's assuming the revision ends in acceptance, which is unknown until the end.

For the CS conferences, there's a guaranteed, no uncertainty, 4 month turn around, and that 4 months includes 1 round of revisions.

So the advantage is pretty clear: a faster and more consistent publication process.

As for the disadvantages you list, I don't really agree with any of them (I won't go into detail why).

I will point out one thing I do think is a disadvantage: the hard page limits imposed by the conferences (typically like 7 or 8 pages, including tables and figures). This is presumably a consequence of the hard deadlines, so that reviewers aren't taken hostage by a long paper. There are a lot of interesting figures made way too small just to meet the page requirement. You also see lots of papers where lots of proofs and details are moved to the appendix. Now this is mostly fine, but sometimes you feel like you need to have two different documents open (the paper and the appendix) and be going between them repeatedly to understand the content. I also feel like it definitely encourages papers to be as brief as possible and avoid exposition, which makes it confusing unless you are already an expert in the subject (which raises the question: how is one to become an expert in the first place?). Lastly, the page limit precludes review type papers.

7
  • 1
    Why can't a faster and more consistent publication process exist for journals? It might not happen for journals in general, but that doesn't mean it's not possible and if that's the only/main reason to favour conferences, it seems like you can just have a journal enforcing the same thing instead to get the same advantage. If journals have tried and failed or there's a fundamental reason why it's not possible, that would be a different story (although it would also then be something to be addressed in an answer claiming it's an advantage).
    – NotThatGuy
    Jun 28 at 0:47
  • 2
    @NotThatGuy The main problem is reviewer reliability. Reviewers don't seem to mind missing the review deadline for a single paper even by multiple months, if it only affects that single paper. In the case of a conference, missing the deadline means that the decision process for the whole conference may get delayed, which puts much more social pressure on reviewers to deliver. Jun 28 at 9:17
  • 1
    One way of making the process more reliable would be to offer better incentives to reviewers for delivering their reviews on time, for example, money. But that seems too revolutionary of an idea to actually happen. Jun 28 at 9:25
  • 1
    @lighthousekeeper: in my experience, the incentive for writing good, timely reviews is... more requests to review. Jun 28 at 12:51
  • 1
    @NotThatGuy There's just no guaranteed fast way to ensure a quality peer review process at the moment with the way it is set up with reviewers, so if you want to get comments on work during peer review, you have to sit back and wait for it (possibly for quite a long time).
    – Tom
    Jun 28 at 12:54
3

This will be a personal perspective, but I hope it sheds some light, First, the fact that CS conferences come in series (usually annual) means that the deadline question isn't quite so limiting. There is always next year's conference.

But, the more important thing, in my experience, is that the periodic meet ups of people in a small sub field or special interest group aids collaboration and synergy.

I travelled to a lot of conferences all over the world for several years. Those get-togethers let a few of us with similar interests brain-storm, ask questions, explore ideas that might be developed, and just reconnect personally, making the ongoing collaboration more interesting. We also got to meet new folks that might be interested in what we were doing.

Some of those conferences (such as those sponsored by ACM) also have special working groups that appeal to a small part of the attendees. Students, in particular, can meet some of the superstars and can be introduced to the members of the working circle of, say, their advisor.

Both the social and intellectual components are valuable in this system. I don't know if we value collaboration more that in some other fields, but the conference cycle certainly enhances it.

I would miss it if it were otherwise. Even though internet communication enables quite a lot of what we do.


Let me address two of your concerns directly.

Tying publications to conferences keeps the flow of new ideas fresh. The publications, coming quickly after results are obtained, gives people something to work with to, hopefully, extend insight and find new potential threads of research. The papers are "more than abstracts" and present actual results that can be built on. Big ideas don't get trapped in a potential multi-year journal acceptance cycle, though some results have been in process for years.

The ACM digital library contains a ton of stuff. It is the go-to resource for conference proceeding for stuff that they sponsor. So, while things can be obscure, not all of the important stuff is.

Some of those annual conferences are held in places where it is difficult for locals to travel elsewhere. This is intentional, to help bring the community together.

2
  • 1
    In my opinion, it would have been limiting if one actually had to wait for the next year's conference if a deadline was missed, but in most subfields there is more than one suitable venue, so one doesn't usually have to wait that long.
    – GoodDeeds
    Jun 27 at 12:23
  • 12
    Thanks! I don't doubt that conferences are useful. All fields have them. What I am skeptic about is tying publications to conferences, a thing which only CS/Eng does. So I don't think the first half of your answer addresses my question (but it might just be because my question is poorly stated). Jun 27 at 12:44
0

tight time constraints in authoring and refereeing actively impact the quality of publications: a paper is not submitted or accepted "when it is good enough", but rather when the deadlines force people to act.

In the field of natural language processing (NLP), rolling reviews have been introduced earlier this year, which hopefully will mitigate this issue: https://aclrollingreview.org/ Also there are a decent amount of conference deadlines throughout the year https://aideadlin.es/ (but admittedly there are some periods lacking deadlines, e.g. ~June 15 to Nov. 15 this year in NLP).

there is a certain duplication of content between conference and journal papers.

The main NLP conferences explicitly state the maximum overlap requirements between the paper submissions and past published work, e.g. (mirror): "In addition, we will not consider any paper that overlaps significantly in content or results with papers that will be (or have been) published elsewhere. Authors submitting more than one paper to ACL-IJCNLP 2021 must ensure that submissions do not overlap significantly (>25%) with each other in content or results".

travel fees (pre-pandemic, at least) add an unnecessary component to publication costs, and raise the entry bar for researchers from some countries.

True but journals often ask for money from either the authors or the readers. They extort billions of USD each year worldwide from universities: Reference for annual journal subscription costs paid per university?. They are some exceptions e.g. Journal of Machine Learning Research, so the issue isn't intrinsic to journals though, just bad old habits.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.