I am from computer science and we typically submit to conferences and less often to journals. Publishing pre-prints to arXiv becomes more and more popular in my field. From the discussion here on AS I get the impression that arXiv is more than just putting something on a personal website.

For example from What to do when you spot a paper on arXiv with the same essential material as yours? I get that I should discuss arXiv papers if they are related to your work.

Now, a paper for a CS conference contains often about 75% theory + 25% experimental section. The experimental section takes a lot of time - so I wonder (and fear) if there is a trend to upload just the bare minimum to arXiv in order to get credit for the idea.

Is this a problem? Maybe similar to patent-trolls, just uploading vague ideas in the hope to get citations?

Edit: For clarification: I am working in the field of data-mining, where it is common to have experiments to show that your idea does not only work in theory. This question/concern came up as we discussed the pro and cons of uploading our work to arXiv, since we have never done this before, but it seems to become more common in our field.

Edit 2: It seems that more people are concerned with this problem, especially in machine learning:

Yoav Goldberg:

This post is also an ideological action w.r.t arxiv publishing: while I agree that short publication cycles on arxiv can be better than the lengthy peer-review process we now have, there is also a rising trend of people using arxiv for flag-planting, and to circumvent the peer-review process. This is especially true for work coming from “strong” groups. Currently, there is practically no downside of posting your (often very preliminary, often incomplete) work to arxiv, only potential benefits.

Yoav Goldberg:

I do not mind posting papers quickly on arxiv. I recognize the obvious benefits of arxiv publishing and fast turnarounds. But one should also acknowledge its shortcomings. In particular, I am concerned about the conflation of science and PR that arxiv facilitates; the rich-get-richer effects and abuse of power; and some of the current arxiv publishing dynamics in the DL community. It is OK to post early on arxiv.

It is NOT OK to misrepresent and over-claim what you did. Sloppy papers with broad titles such as “Adversarial Generation of Natural Language” are harmful. It is exactly the difference between the patent system (which is overall a reasonable idea) and patent trolling (which is a harmful abuse).

[...]

Most people don’t read the papers in depth but only the title and sometimes the abstract and sometimes the intro. And when the papers come from established groups, people tend to trust the claims without verification. “Serious researchers” might not fall for this, but the general population sure does get mislead. And by the general population I mean people who are not actively working in this exact sub-field. This includes practitioners in industry, colleagues, prospective students, prospective reviewers of papers and grants. In the short time since this paper came out, I already heard, on several occasions, “oh, you are interested in generation? have you tried using GANs? I saw this recent paper in which they get cool results with adversarial learning for NLG”. This will be extremely harmful and annoying for NLG researchers who apply for grants in the coming year (remember, many grants are reviewed by a panel of capable but non-specialized experts), as they will have to either waste precious space and effort in dealing with this paper and with Hu et al and explaining why they are irrelevant, or be dismissed as working on this “already solved problem”, despite the fact that neither the paper in question nor Hu et al actually did very much, and despite the fact that both papers have terrible evaluations.

And the follow-up discussion on reddit

  • 7
    People are not stupid --- if you produce lots of half-baked papers, they notice. It is similar to normal conferences/journals, where you can churn out Least Publishable Units if you wish, but would be penalized for doing so. – Boris Bukh Jul 25 '15 at 12:20
  • 1
    @BorisBukh - Could you please elaborate on the penalization part? Exactly how? – The Dark Side Jul 25 '15 at 14:36
  • 2
    @TheDarkSide You get a bad reputation, so people are less interested in working with you or hiring you. – David Richerby Jul 25 '15 at 15:50
  • 2
    I know this is not the actual question, but this 75% + 25% breakdown is far from universal for CS. Papers in conferences I usually go to contain 99%-100% theory. For the actual question, you can be sure that the first person to develop the idea and demonstrate its usefulness will get at least as much credit. – Sasho Nikolov Jul 25 '15 at 18:53
  • 3
    @ManuelSchmidt First, if people think you did not do the work, they won't give you credit. Second, most half-baked ideas are no good --- the chances are that, if you do not actually do the work, you will be generating lots of rubbish ideas. Even if there is one-two sensible idea there, no one will take you seriously. – Boris Bukh Jul 25 '15 at 19:51
up vote 23 down vote accepted

It seems to me that this question is less about the arxiv per se and more about how to navigate doing research in a very fast moving academic field.

I get the impression that arXiv is more than just putting something on a personal website.

It's certainly different. The main differences are:

(i) Many more people will see your paper.
(ii) Your paper will indeed be archived, essentially permanently. (Withdrawing a paper from the arxiv has the effect of uploading a new, empty version. Older versions are still there!) On your own website, you can take things down at least as quickly and easily as you can put them up.
(iii) Some (very obnoxious) journals may regard posting on the arxiv as "prior publication". (This is strictly unheard of in my field, mathematics. My guess is that CS is close enough to math so that it is at least very rare in yours.)
(iv) Minimum standards of completeness and professionalism are enforced on the arxiv. These are enumerated on the site itself, but the gist of it is that they are looking for manuscripts at the last step before conference/journal submission or later. They are not looking for early drafts.

Of these points, probably the last is most relevant to you. If it is standard in your subfield to include 25% experimental data [you say that is standard in "CS", but that is certainly not true across the entire field], then a paper uploaded to the arxiv without that would probably look to many in your field to be incomplete, which is against the spirit and perhaps the rules of the arxiv. So I wouldn't recommend it.

But the situation doesn't fundamentally change for papers that you or others post on your own website. The phrasing in your question suggests that you feel that you might not have to "be responsive" in the academic sense to papers that you find on people's webpages (only). That's not true. As an academic you have to be responsive to others' work wherever you find it.

In terms of the prospect of people uploading "the bare minimum to arXiv in order to get credit for the idea": is this an actual problem for you or just something you are wondering might be a problem? I have never encountered this problem in my work. That you are wondering whether it might be a problem makes me think you may be a quite new researcher and haven't fully grasped the way the academic community works. (Which is fine, and you have only to look forward to understanding it better. But you should talk to others, including advisors and mentors, to try to get a better idea.) Academia places a great privilege on completed work for exactly this reason. If you put out a manuscript which, say, modifies an algorithm and hints that it could be faster in some situations, the most likely reaction you'll get is "Go on..."

This question may finally have made me understand what people on this site are on about when they say things like "An idea is worthless". An idea is certainly not worthless, but a vague and unimplemented idea is of highly uncertain value, to the point where rushing to publish "only the vague idea" would be a very poor, um, idea.

By the way, you don't have to immediately drop something because someone else had "the same idea" and put out a paper before you. Much -- perhaps most -- important academic work overlaps with other work and even more of it refines and extends the ideas of others. How to respond to seeing "your idea" in another paper is a topic for a different answer.

Finally, let me say: if what you've done, are doing or want to do has real value, then it is unlikely to be received with thunderous applause this week and totally ignored next week. If you're living in fear that someone else will say what you want to say, maybe slow down and find more to say.

  • 3
    This is an excellent answer that avoids pejoratives, name-calling, and too much attribution of motives to the OP that may or may not be there. +1 – Bill Barth Jul 25 '15 at 19:15

If there is a trend to upload just the bare minimum to arXiv in order to get credit for the idea.

No, there is not. No reputable author is stupid to publicly upload junk (on arxiv or anywhere else) and put his name on it. Perhaps, there are some people that do what you are suggesting (I cannot confirm though) but these mediocre-to-bad publications are nothing you should worry about.

  • 4
    There's no evidence that OP was talking about uploading half-baked or junk-quality things to the arXiv. I read this as saying "I've got the theory section of a paper mostly done, but I haven't done all the experiments required for a full journal/conference article, should I upload the first half of the paper in order to stake my claim? Are others doing this such that I should be worried about getting scooped if I don't stake a claim" – Bill Barth Jul 25 '15 at 16:15
  • 1
    @BillBarth. In certain CS disciplines (e.g., experimental algorithms) uploading an idea and saying: "it will work because I say so, with no actual experiments to prove it" qualifies as junk. So, a) the OP should not do it for his work or b) worry that someone else does it, because no one pays attention to such claims with no experiments. – Alexandros Jul 25 '15 at 16:37
  • 2
    I would think that a proof of the computational complexity of a new algorithm, for example, would be a non-junk half of a paper that might then want some experiments to go into the full article. I don't see anywhere where OP has asked us to trust them. I think your commentary here is too harsh and too absolute. You could soften this answer substantially and simultaneously improve it markedly. – Bill Barth Jul 25 '15 at 17:28
  • 3
    @Bill: I agree with you that the core of the question seems to be: "Should I cut corners in my work in order to get it out first and stake a claim?" This is a question for the OP to ask mentors with experience in his field. However, as a general answer i would say "No. On the contrary, you should take the time to make your work as strong as possible. Releasing work that you know is seriously incomplete has every risk of diluting or ruining the impact of your work and your own reputation." – Pete L. Clark Jul 25 '15 at 18:39
  • 1
    I would think that a proof of the computational complexity of a new algorithm, for example, would be a non-junk half of a paper — Sure. But OP specifically suggested "upload[ing] just the bare minimum to arXiv in order to get credit for the idea". Proposing and analyzing an algorithm is far from the bare minimum; indeed, most of my papers do nothing else. – JeffE Jul 25 '15 at 23:18

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.