As it can be appreciated from this list of journals with varying preprint policies, certain journals consider a preprint to be "prior publication". In other fields like Chemistry, there is a strong policy against preprints.
I'm curious about those reasons, if there are other reasons, and if they hold weight.
A a chemist, I'm very well aware of this.
Here's the ACS Journal Editors' Policy on Preprints' point of view about the disadvantages of preprint servers:
The disadvantages of preprint servers include: the potential for flooding the literature with trivial and repetitious publications, thus making extraction of reliable and valuable information more difficult; absence of peer review; possible premature disclosure with inadequate experimental details or supporting data; premature claims of priority; potential lack of proper references and credit to prior work; abuse of multiple revisions or updates; possible lack of duration and long term archiving.
Personally, I find the two concerns about
the most relevant points.
There have been "experiments" with preprint servers for chemistry some 10 years ago [1] but AFAIK they did not develop the momentum e.g. arXiv has, and they seem to have died meanwhile.
See also: Cecelia Brown: The Role of Electronic Preprints in Chemical
Communication: Analysis of Citation, Usage, and Acceptance in the Journal Literature, Journal of the American Society for Information Science and Technology 54.5 (2003): 362-371.
(the discussed server seems to be down - or at least I can't get a response).
The possibility to be able to publish a manuscript on a preprint server before submitting it to a journal is not as imortant for me personally as the possibility to make the final contents of the paper publicly accessible.
Thus I can live quite well with not being allowed to submit manuscripts that are already available on preprint servers as long as I'm allowed to also publish the manuscript (preferrably the final version after peer-review) after I submitted it to the journal.
One key issue to keep in mind when comparing different fields is the scale of money involved. For example, according to their financial statement, in 2012 the American Chemical Society received $421 million in revenue for electronic services, including both journals and the Chemical Abstracts Service. That's a staggering amount of money for a scholarly society. (For comparison, the American Mathematical Society's 2011 revenue from Math Reviews and journals was $15.5 million.) The ACS is the gatekeeper for publications and data that are worth a fortune to industry, so they have a powerful incentive to maintain that control. It's no coincidence that they are much less friendly towards open access, the arXiv, etc. than corresponding groups in mathematics or physics are.
There are commercial reasons for journals to be the only place where the article can be obtained. (advertising on the site or in the print journal). So simply violating their policy (if stated clearly) is one valid rejection reason.
(I personally disagree with this reason but such is life)
As said in comment, one reason not to allow preprint publication alongside journal publication is to preserve the incentives to subscribe. To add a note to this point, let me remark that most of the preprint-friendly publishers (this adjective includes Elsevier and Springer: they don't do everything wrong) do not allow the final journal-template version of the paper to be deposited in an open repository. In other words, most publishers do forbid open distribution of published papers in some way, they draw the line at different points. Of course, drawing the line after or before the preprint version of the article makes the most important difference.
Another reason in some field, alluded to in the question, is a way to understand the pretty general policy that journals' goal is to publish novel publications. In all fields, this notably means that you are not allowed to submit to a journal a paper that has already been published. In some fields, notably humanities fields (at least in France) this extends to journal refusing to publish articles already available as preprint. As far as understand, preprints are then really seen as publications, in the sense that they are no more novel. Of course, they are not considered as publications in the same way than journal articles in CVs...
Concerning the weight of these reason, it feels to me like tradition has a lot to do with it. Some tradition are easier to sustain in some fields than others; e.g. Chemistry can ask both reader-side for subscription charges and author-side for pages or color figure charges, as the field has some money notably due to its experimental nature; such tradition would be more difficult to sustain in humanities where money is much scarcer. As another example, it might be that the strong weight of book publishing in humanities is related to this "preprint is prior publication" point of view: it is more common for publisher not to allow books to be made available, and a field where books bear at least as much importance as article for idea dissemination seems more likely to adopt the same policy for articles.
