I am from computer science and we typically submit to conferences and less often to journals. Publishing pre-prints to arXiv becomes more and more popular in my field. From the discussion here on AS I get the impression that arXiv is more than just putting something on a personal website.
For example from What to do when you spot a paper on arXiv with the same essential material as yours? I get that I should discuss arXiv papers if they are related to your work.
Now, a paper for a CS conference contains often about 75% theory + 25% experimental section. The experimental section takes a lot of time - so I wonder (and fear) if there is a trend to upload just the bare minimum to arXiv in order to get credit for the idea.
Is this a problem? Maybe similar to patent-trolls, just uploading vague ideas in the hope to get citations?
Edit: For clarification: I am working in the field of data-mining, where it is common to have experiments to show that your idea does not only work in theory. This question/concern came up as we discussed the pro and cons of uploading our work to arXiv, since we have never done this before, but it seems to become more common in our field.
Edit 2: It seems that more people are concerned with this problem, especially in machine learning:
This post is also an ideological action w.r.t arxiv publishing: while I agree that short publication cycles on arxiv can be better than the lengthy peer-review process we now have, there is also a rising trend of people using arxiv for flag-planting, and to circumvent the peer-review process. This is especially true for work coming from “strong” groups. Currently, there is practically no downside of posting your (often very preliminary, often incomplete) work to arxiv, only potential benefits.
I do not mind posting papers quickly on arxiv. I recognize the obvious benefits of arxiv publishing and fast turnarounds. But one should also acknowledge its shortcomings. In particular, I am concerned about the conflation of science and PR that arxiv facilitates; the rich-get-richer effects and abuse of power; and some of the current arxiv publishing dynamics in the DL community. It is OK to post early on arxiv.
It is NOT OK to misrepresent and over-claim what you did. Sloppy papers with broad titles such as “Adversarial Generation of Natural Language” are harmful. It is exactly the difference between the patent system (which is overall a reasonable idea) and patent trolling (which is a harmful abuse).
Most people don’t read the papers in depth but only the title and sometimes the abstract and sometimes the intro. And when the papers come from established groups, people tend to trust the claims without verification. “Serious researchers” might not fall for this, but the general population sure does get mislead. And by the general population I mean people who are not actively working in this exact sub-field. This includes practitioners in industry, colleagues, prospective students, prospective reviewers of papers and grants. In the short time since this paper came out, I already heard, on several occasions, “oh, you are interested in generation? have you tried using GANs? I saw this recent paper in which they get cool results with adversarial learning for NLG”. This will be extremely harmful and annoying for NLG researchers who apply for grants in the coming year (remember, many grants are reviewed by a panel of capable but non-specialized experts), as they will have to either waste precious space and effort in dealing with this paper and with Hu et al and explaining why they are irrelevant, or be dismissed as working on this “already solved problem”, despite the fact that neither the paper in question nor Hu et al actually did very much, and despite the fact that both papers have terrible evaluations.
And the follow-up discussion on reddit