27

I was speaking with several professors in my department about submission to high-profile conferences like CHI in human-computer interaction. Most of the professors were of the opinion that acceptance in most of these high-profile conferences (typical acceptance rate of less than 20%) is heavily influenced by politics: even though you may have comparatively good research, the editors will most likely be more favored toward accepting papers published by Microsoft Research or Google Research or from other high-profile universities where they have close relationship with the faculty and researchers.

The professors advised me against spending time and resources on publishing to these conferences and instead concentrate on second-tier journals (like those published by Springer) for better return on investment for my efforts.

I assume in good faith that most of the editors may not reject papers without reading them, but does anybody concur with the opinion of professors at my department?

33

It is an often-heard prejudice in CS that many of the top conferences are relatively closed "boys clubs". Indeed, if one just looks over accepted papers for multiple years, one typically ends up seeing the same affiliations over and over again, strengthening this impression.

However, this could be due to a number of (good or bad) reasons:

  • Papers from professors / labs that are well-known in the field may simply be of much higher technical quality, making it quite natural that they also get accepted much more often.
  • Professors / labs that are well-known in the field generally know how to write papers for this specific conference. They know what the TPC values, and how to present their results in a way that is appreciated in the field. As these professors are typically in the TPC themselves, they know what kind of papers usually get accepted and which are rejected.
  • Professors / labs that are well-known in the field often have a better grasp on the existing state of the art in the field, making it easier for them to identify what is good and novel. In my experience, "outsiders" have a tendency to overestimate the novelty of their contributions significantly. Further, well-known labs know what problems are currently en vogue in the community.
  • It is of course also perceivable that papers from well-known professors / labs are just not judged as critically. For instance, a reader may very well think that a paper is not applying a given technique correctly, but as the paper comes from the group that invented this technique, he gives them the benefit of doubt and assumes that they will know the technique better than him. He would probably not extend the same favorable thinking to an outsider.
  • Finally, for some topics, it is just easier for some labs to do good research than others. A common example are the web search tracks at the WWW conference. These tracks typically require the validation of new algorithms on real data, to which mostly only industry labs from Yahoo! etc. and their close collaborators have access.

Note that none of these reasons is really politics. Indeed, I would argue that all of the reasons above are significantly more likely than a paper getting rejected for the reasons you cited.

I assume in good faith that most of the editors may not reject papers without reading them

I would say, at a top conference such as CHI, you can rely on your paper at least getting reviewed thoroughly, yes.

but does anybody concur with opinion of professors at my department?

Well, given all the reasons above, it is indeed quite likely that your paper will be rejected. By definition, if a conference has a <20% acceptance rate, rejection is always a real possibility (clearly, it happens to most submissions). However, I am wondering why it would be better to not even try if you think your work is good enough for CHI. If the paper gets rejected there, you can still re-submit to a lower-tier venue, and you receive a number of hopefully helpful reviews. The only disadvantage I see is that it prolongs the publication process by half a year, but if you see any chance of the paper begin accepted at the top-tier venue, I think this should be worth it.

  • 8
    I agree; the attitude expressed by the professors seems overly defeatist. Whatever happened to the "college try"? – J.R. May 2 '14 at 8:30
  • @xLeitx: in your last paragraph you state: > Well, given all the reasons above, it is indeed not unlikely at all that your paper will be rejected. This triple negative was difficult for me to parse. I do understand that it is not the same as stating "... it is indeed likely that you paper will be rejected" and also not the same as "... it is indeed not likely your paper will be accepted" but isn't there another way to phrase it? – user13588 May 2 '14 at 15:22
  • One final aspect that is not mentioned here is the issue of scoping for a conference. Reviewers tend to give extra points for things that they are interested in (which makes sense). However, this causes a feedback loop where conferences tend to get more and more restrictive in scoping. In an interdisciplinary conference that I'm involved with, it's a constant battle to keep a variety of research areas well-represented, because if the balance of reviewer interests goes too far in one direction, good research on the other sides gets marginalized (e.g., excellent papers relegated to posters). – Namey Sep 7 '15 at 20:26
  • Good answer. End of the day, some places are simply resource rich. They have talented and experienced people, easy access to data and funds, and a supportive environment. If you don't belong to these places, it makes it harder or your idea must be truly outstanding or outshine those from big league players. – Prof. Santa Claus Apr 24 '17 at 6:02
  • 1
    For what it's worth, none of the reasons listed here seem overly "bad" to me. Even #4, which is slightly quesitonable at first sight, is based upon the assumption that "the group that invented [a] technique (...) will know the technique better than [the reviewer]" - and that assumption, for better or worse, is probably simply true. – O. R. Mapper Apr 24 '17 at 6:09
15

Having served on the CHI program committee, many other program committees, and having been program chair of a couple of other HCI conferences, I think your professors are overly cynical.

First, as JeffE pointed out, CHI (and many other conferences) use blind or semi-blind reviewing, so at least the reviewers do not know whose paper they are reviewing.

Second, I have never been in a PC meeting where the identities of the authors was a point of discussion. Of course that doesn't mean that the associate chair (AC) for the paper was not influenced by who the authors were, but it would certainly imply that there is no institutionalized bias for or against particular authors or institutions.

I would definitely not worry about your papers being rejected without being reviewed; in fact, when you submit you will get the reviews so you know why your paper was or was not accepted.

  • 2
    While I think this is all true, to play devil's advocate, I would say that anyone who knows a field well can often infer which group (if not which authors) wrote a given paper. There are just certain research topics or strategic advantages (e.g., proprietary data sets) that groups are known for. I agree that I haven't seen this impact decisions though: nobody cares who the group or researcher is. – Namey Sep 7 '15 at 20:21
5

I completely agree with almost everything xLeitix wrote, except the conclusion. One thing that was not addressed is why your professors told you not to submit to CHI. It's possible that they're giving their honest opinions, or that they themselves had a bad record submitting there.

But it's also possible that they are telling you that your work is not good enough for CHI, but in a gentle manner. Also, they are familiar with where you are in your career track and may believe that losing time and effort on a failed CHI attempt is not good for you. Both of these require perspective that you cannot have on yourself.

Consequently, I would suggest that you take the advice you were given.

  • 9
    But if your professors think your work isn't good enough for CHI, they should say so directly. Being "gentle" is not actually doing you any kindness. You deserve an unvarnished appraisal of your work. Demand it. – JeffE May 2 '14 at 15:20
4

This kind of advice is often given to people, but in my view it's fundamentally misguided.

It's true to some extent that many of the top conferences have a particular way of doing things, and that people in the community associated with a conference will often have more experience with that way of doing things and find it easier to get papers accepted there. However, it's untrue that the communities associated with particular conferences are a closed shop: it's entirely possible to adapt your way of doing things to what you know different conferences expect, and that will improve your acceptance rate at those conferences. Moreover, it's absolutely worth your while to do this, because publishing at the top conferences massively improves your chances of having a successful academic career (not to mention that it puts you in contact with a large number of good researchers in your field).

In that sense, the people who are suggesting that you eschew the top conferences and simply send your work to second-tier venues are doing you a serious disservice, because they're basically ensuring that you're not even on the pitch where you could compete with your peers (indeed, the heavy focus that top groups put on publishing at top conferences tends to mean that they are underwhelmed by people who don't compete). If you're good and you put in sufficient effort, you can definitely play at the right level, whereas counting yourself out of the game before you've even tried to succeed is deeply unwise in my view.

TL;DR The people giving you this advice are leading you astray in my view. It's not wise to spend your time and energy submitting papers to venues that many researchers will view as being less than top-flight, when you could be refining your papers to make sure that they get into the top venues.

More generally, people unfortunately tend to give similar advice about all kinds of things, not just academic conferences. A classic example would be applying to university, or applying for top jobs. The general form of such advice tends to be:

  • This venue is biased against you specifically, or against people like you (whatever that means) in general.
  • This venue wouldn't suit you if you were accepted -- you'd enjoy it way less than a lower-tier venue.
  • This venue isn't all it's cracked up to be anyway -- they just have a high opinion of themselves.

There are certainly situations in which people might be a bit biased, or in which you wouldn't enjoy some place, or in which the place isn't as good as it thinks it is, but these things are by no means universally true. If people tell you "don't apply to X because I have specific, verifiable evidence of Y", then that may be one thing (check the "verifiable evidence"), but if they tell you not to apply to top venues in general, then you need to seriously question that.

  • 1
    Another good answer. My only gripe here is that there is a whole world (so to speak) of conferences between the high-profile conferences with acceptance rates around 20% mentioned by the OP and "venues that many researchers will immediately discount". Not going for the flagships does not automatically mean turning to the "conference-equivalent of predatory journals". – O. R. Mapper Apr 24 '17 at 8:57
  • @O.R.Mapper: Yes, true, I wasn't trying to imply that it meant that (I've edited it accordingly). Nevertheless, many jobs do mention publications at specific top conferences explicitly, and publishing at second-tier (though non-predatory) conferences will definitely help you less for those kind of jobs. – Stuart Golodetz Apr 24 '17 at 12:58
3

I agree with xLeitix's answer and would like to elaborate on how some high-profile researchers manage to constantly appear at certain high-profile conferences.

I'd say the method to reliably appear on a given conference are resources and focus:

First, to professor X, conference A is not simply another conference. It is their subfield's "flagship" conference, their "home" meeting. They will and can put a lot of effort into making sure they get published there. Thus, they do not just submit one paper. They make sure they have at least a short paper about each of their hottest ideas to submit. With ten or more submissions, a few acceptances are likely even at a low acceptance rate.

Related to that, note that the conference topic is probably not just X's personal hobby. More likely, they have an entire research group specialized on the conference's core topic at their disposal. X is probably already involved with half a dozen projects with different members of their group at any time, all of which are thematically suitable for conference A.

Lastly, there is simply a strong overlap between X's topics of expertise and A's topics. Even if X does, somehow, not manage to publish in A, they will actively participate nonetheless (thereby adding to the impression that their name appears all the time with respect to A):

  • They may be among A's chairs.
  • They might chair one or more sessions at A.
  • They might be invited to give a keynote talk at A.
  • They might offer a workshop co-located with A. Note that even without anything questionable going on, this could lead to a certain additional bias towards contributions by X's group: While less frequently so for workshops at very large conferences, workshops often rely extensively on word-of-mouth marketing. As a consequence, people in X's direct vicinity are most likely to know about the workshop and its precise focus.
  • As a well-known member of the audience whose work is being built upon by other submissions to A, they might be mentioned in some of the talks. This can happen in totally informal remarks: "Last year, Alice, who is sitting over there, presented technique Y and asked Z, so I have been looking into Z."

And even if all of these fail, X will still attend A and attract enough attention during coffee breaks to reinforce the memory: X was actively part of A again.

1

Politic is everywhere, of course, especially in situations where the resource giving out to people is limited. Bear in mind, taking a look at the program committee before submitting! And please don't stick to one conference! Even some well-known professors get rejection at some conferences, let alone others.

-5

I concur. Though from personal experience it is hard to figure out whether the good conferences are giving you a fair chance per se, it is important that defined timelines decide where you present the papers and complete your qualification requirements and showcase your research.

I am fairly certain no times could be spent on review of individual papers before selection, so one must simply choose conferences in one's orbit and rely on the nominal Gravitation your mass possesses, than engender new attraction and target farther constellations for a first space flight.

However, on the point of this being causal to politics or from politics, a big No. Just that no one gives it time enough to read and even when one does , choosing between comparable efforts becomes pertinent on the research direcion of the day according to the reviewer and the candidate's background. So the conferences just choose from the nearby villages / planetary systems as the monitoring will be a black hole and even sincere reviewers without politics will just choose the simplest and best research which will happen to be less confusing if it is from the neighbourhood you already know.

  • 2
    I am fairly certain no times could be spent on review of individual papers before selection — If I am parsing this sentence correctly, you are wrong. Reviewers for most flagship conferences actually spend a significant amount of time reviewing submissions, and program committees spend significant time comparing strengths and weaknesses of different submissions, before any decisions are made about which papers to accept. – JeffE Apr 24 '17 at 10:13
  • 2
    This answer deserves to be down-voted solely for the overstrained space-flight metaphor. – JeffE Apr 24 '17 at 10:16
  • @JeffE I have made 8-10 confernces and failed at 2-3. I am fairly certain they are combing through the papers only at the AFA one which is currently 'undergoing selections' and they probly will at the Cavalcade come December (Asia) Else nichts, no one is combing through them. No a single review report. – Suntropical Apr 24 '17 at 18:19
  • @AmitMittal I have no idea what "AFA" or "the Cavalcade come December (Asia) Else Nichts" are supposed to mean, but if you really are not getting review reports at all, then either you're submitting to crappy conferences or you're not submitting to computer science conferences at all (like CHI; see the original question). – JeffE Apr 24 '17 at 19:51
  • No @JeffE I am nit from CS and am answering only as an Academic. I am doing my PhD in Finance. glad to sort that out. – Suntropical Apr 24 '17 at 20:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.