87

In the field I specialize in (I'm currently a moderator of crypto-SE), I see many obviously pointless papers published, often in publications where that costs money. I imagine that's the same in many fields. Why are such papers published, or worse, studied?

Example: there are tens of thousand papers on image encryption, including thousands at MDPI, which has Article Processing Charges. Yet, the subject is almost¹ enough to tell the work is pointless: once digitized, and especially after it's compressed (as almost all images are today), image is data that can be encrypted just like anything else. The International Association on Cryptographic Research correspondingly publishes no paper on image encryption in its peer-reviewed publications; many of the papers about image encryption it references are refutations, and they do not go further than its non-reviewed preprint server.

Worse, I regularly see evidence that students at least think they have no choice but study a pointless paper riddled with errors. In this recent example, the paper attempts to apply an (unstated) Elliptic Curve signature scheme to inter-vehicles communications, but examination shows that the authors failed to find parameters for a usable Elliptic Curve. The paper cites 36 references on the same vein, many comparably poor. The OP ended up writing (in chat) "unfortunately I have no other choice but to study this paper", and I could not determine for sure if it was designated to them as a reference, or was just a poor choice.


Addition: what about the hypothesis that some subfields like digitized image encryption are abandoned as obviously not worth attention by actual reviewers, and flourish precisely for that reason? This particular subfield is remarkable in that it has developed standard patterns for the many articles, and some bogus measures of the efficiency of the encryption. To see what I mean, sample the papers this query returns (at least these all are officially free to read, if not to publish).


¹ I admit we must check that the image is encrypted and decrypted by a computer, and that the sole objective is encryption (rather than e.g. hiding data into an image, that is steganography, or Thumbnail-Preserving Encryption). Examination shows that most articles returned by that query are only about encrypting a digitized image, and many of the others are seriously defective.

7

12 Answers 12

115
  • Some poorly-run institutions pay people to publish papers. The pay is not based on the quality of the papers, only the number and possibly other useless information like the indexing of the paper.
  • Some poorly-run institutions require students to publish papers to get degrees, but the quality of the papers is not adequately assessed.
  • Some people do not realise their papers are pointless.
  • Some of the papers that are pointless in your opinion are not pointless to other people.

There are also many good papers.

12
  • 8
    Editors should filter out most of the garbage, I hope. Unless the journals are either predatory or very low quality.
    – Buffy
    Dec 27 '20 at 12:45
  • 56
    I would also mention bibliometrics, personally: some people and institutions use number of papers and citations to judge research output, for tenure, promotions, and funding. Dec 27 '20 at 13:10
  • 19
    @Buffy I strongly disagree; scientific research should be judged by editors on the validity of the methods, not on the "point" of the paper. Otherwise you end up publishing mostly statistical outliers instead of the truth. Other fields might be different. Dec 28 '20 at 0:31
  • 14
    "Some of the papers that are pointless in your opinion are not pointless to other people." that's the key point here.
    – Gimelist
    Dec 28 '20 at 4:59
  • 10
    @Gimelist: shouldn't there be a limit to that? A good fraction of the papers I'm talking about are not only pointless, they affirm falsities, and in my opinion obviously so for any sane person having studied crypto for a year. Nobody knowing crypto has advised the authors, or they didn't listen.
    – fgrieu
    Dec 28 '20 at 9:12
80

I used to laugh about a pair of pointless papers. The authors had (independently) proved that Polish notation not only makes formulas of propositional logic unambiguous without needing parentheses but continues to do so if the formulas are written in a circle, so that one can't immediately see where the beginning and end of the formula are. Later, I learned that this result provides a key step in some combinatorial arguments, including a proof of the Lagrange inversion theorem for formal power series. (The people who provided that proof of the inversion theorem were apparently unaware of the connection with logic.) The moral of the story is that I should not have judged pointlessness by the original purpose of the work but should have taken into account possibly unexpected applications elsewhere.

2
  • Comments are not for extended discussion; this conversation has been moved to chat. Please read this FAQ before posting another comment.
    – Massimo Ortolano
    Dec 30 '20 at 12:34
  • 1
    Yes. I felt the same way 30 odd years ago when reading a paper on the mechanical properties of potatoes, apples and blades of grass in Acta Materialia. Of course I was very wrong as nature evolves many structurally-efficient materials - something a few of our new materials scientists have learned from. But perhaps the OP had dumb rather than apparently inapplicable papers in mind . . .
    – Trunk
    Dec 30 '20 at 17:59
45

Let me propose two counterpoints to your question.

First, what one finds silly someone else doesn't. I'm amazed that my most cited paper is basically aspirational and does not contain any strong result; on the other hand my best work is not cited very much at all. In fact there is usually an inverse correlation between my ranking of my own work and the number of citations. Hence what I find sillier others do not.

Second, I realized some years ago that some papers are just "silly" because they are - in some sense - practice. I (or a collaborator or a student) had to publish papers under tight timelines, or under some pressure for a grant or something other deadline. As a result, these are compromise. I used to worry quite a bit about this until I visited the Musée d'Orsay in Paris: there you will find that, before making their "big work", the masters practiced on "études", a series of smaller tableaux where only some small elements are different. It all looks very incremental. There is one series in Orsay where three "études" show the same haystack with different hay colors, different backgrounds etc.

Now... I don't want to defend people who constantly and only publish "silly" papers, but I will add the following anecdotal evidence: a lot of the more incremental stuff is just to keep you going to a better, more properly formed, final non-silly paper. I see this quite a bit of that as a referee for journals or grants: there are 3 or 4 papers in a series, and they eventually open the way to something more substantive and unexpected.

Sometimes not... you realize that the small "silly" papers already comprise all there is to say; the better researchers will move on.

6
  • 7
    This is definitely my experience. I am growing as a researcher as I publish more papers, and looking back, my first papers are honestly complete trash -- especially in comparison to my most recent. People will probably think my first papers are 'silly' and I hope I can say the same about this most recent paper in a few years time.
    – The Dude
    Dec 28 '20 at 17:21
  • 2
    When you're hunting for a citation for a fact you "know" is true (or at least want to believe) it's much easier to cite aspirational papers. I wonder if this explains your experience?
    – alessandro
    Dec 29 '20 at 1:38
  • @alessandro I believe you are correct. There’s even a name for this: proof by authority ( see en.m.wikipedia.org/wiki/Argument_from_authority). Dec 29 '20 at 2:12
  • 2
    @DKNguyen One that makes claims (sometimes bold) that are prima faciae true but without supplying solid evidence to back it up. Dec 30 '20 at 0:11
  • 2
    @ZeroTheHero I think that I'm also guilty of a number of aspirational papers - although I think my reasoning was faultless - since detailed confirmatory studies were not made ! But don't forget that those who spend too much time gathering experimaental data to support their envisaged explanation run the risk of having their "scoop" hijacked by rival researchers, viz. Rosalind Franklin's elaborate XRD work on DNA structure being hijacked by Watson.
    – Trunk
    Dec 30 '20 at 18:10
24

In my view, there are three main reasons:

  1. Academia is an economic institution where money flows in and papers flow out. Just in the US, the annual budget of NSF, NIH, DARPA, DOE and many others exceed the GDP of a mid-sized country. The pace of breakthroughs cannot keep up with the amount of resources we allocate for research but researchers need to produce papers as deliverables of their funding. Remember the old joke of getting 9 women to deliver a baby in a month.

  2. "Pointless" papers are needed "as fronts" to enable researchers to continue working in a field for many years. They may also enable high quality work in serendipitous ways. I do not think there can be a model where all papers are high quality without substantial changes. You may not agree with the boundary conditions of such a model either: Imagine arguing science takes time and try democratizing research funds by taking the competitive process out.

  3. There is usually no incentive for researchers to optimize the quality or the individual contributions of a single paper: As long as grants come in, especially from government-funds like NSF, there is little control on the quality of the output.

4
  • 1
    In 2, are you suggesting the people writing the silly papers on image encryption I'm talking about are actually working on something else that's serious?
    – fgrieu
    Dec 27 '20 at 21:53
  • 4
    People who are writing "silly papers" on image encryption are not writing one paper over the course of their career. If you look at the output of a research lab over a course of a period you'll see that the quality of contributions will vary. People usually work on multiple projects at the same time, some of them may look "sillier" than others. The examples you are finding may not be representative of the overall contributions of an individual or a group.
    – anon248
    Dec 27 '20 at 22:08
  • 4
    Referees of a reputable funding agency will not be impressed by crappy papers in low-quality journals. On the contrary, they are going to cut the funding of the respective projects at the next possibility.
    – Karl
    Dec 28 '20 at 12:26
  • 3
    Not sure what you mean by a "reputable funding agency". As I have noted, many of the funding agencies do not have a mechanism to check what appears in the wider literature as a result of their funding. This is partly the reason there are so many seemingly pointless papers. You may be talking about idealities, I am talking about the current praxis.
    – anon248
    Dec 29 '20 at 0:55
15

This is similar to other answers, albeit framed in a different way.

  1. 'Publish or Perish': At least in Europe, an academic's career is intimately tied with publications and you're expected to regularly churn out papers. To an external observer, the quality of the paper is usually tied to the journal ranking rather than the actual content itself. If your paper doesn't get published by a reputable journal, you can still get it out through other means (I'm sure many of you know what I'm talking about here). Hence we flood the internet with lower quality publications, making it exceedingly laborious trying to find the relevant gems.

  2. 'Journal publication as a business': The existence of high quality peer-review is integral in furthering science. However this viewpoint doesn't appear to be shared by all and hence we end up with publication houses that try to muscle in to get a piece of that pie.

  3. 'Information overload': The internet is rife with noise and this is also reflected in academic publications. Too much information is akin to white noise that drowns out the truly relevant information. You can see how #1 and #2 affects research in this regard. It's a vicious feedback loop.

I agree with the answer from ZeroTheHero in that it's best to look at it as a refinement process and consider the lower quality publications as incomplete works.

1
  • 3
    Agreed, even if you know your paper is bad or even wrong, there is simply very little incentive not to publish. It’s kind of good that your reputation isn’t ruined forever if you publish bad papers, but it leads to the kind of “noise floor” you mention.
    – Michael
    Dec 29 '20 at 14:34
11

I'd add one point to the existing answers: Training

Somehow you need to train your bachelor, master and PhD students to write technically good papers. Often they will get some toy topic, that could show good results but might also be just a bit crunch work or a "silly" approach that helps them get into the topic and learn to do proper research (including scientific writing). Obviously someone higher up the chain will assist them and thus also be on the paper. And then you want to publish it, so if you know it is low level impact, a bit weird or "pointless" (e.g. very much expected results, albeit not done exactly like this), you look for a low-level journal and place it there. That can be just a low reputation journal that takes all that is not fake. In that case it's not about getting a good publication to brag about or to have a large scientific impact. It is just (mainly) so your students go through the process of refining and handing in the paper once. It also helps them to bolster their ego, and again to do the ground research in that area, that hopefully enables them to pick more specific more interesting problems and more reasonable approaches etc.

4
  • 1
    I don't see any valid argument here, FH. The problem most students have is in writing simply, clearly and to the point about their work. Most of the papers you read in journals are exercises in obscurantism in both content and style, i.e. saying nothing about vital experimental techniques and going on at length about esoteric points - all in an archaic writing style. You just won't get any pointers on this from academic papers. Better to go to textbooks. As to the point of structuring your paper properly: all well but you MUST have something new to say first.
    – Trunk
    Dec 29 '20 at 13:14
  • 4
    There may be something to this, but there are several caveats. First of all, in my field (theoretical mathematics) it hurts, not helps, to have an evidently pointless paper, and it would be especially strange to see a senior coauthor on it: they should "know better." Second: I don't think someone gets good training from writing a pointless paper: as the other commenter says, having something new to say is indeed the most important part of writing a paper. Dec 29 '20 at 19:48
  • @PeteL.Clark my point is not that it helps your career by getting citation bonus points, but that it helps students to become better writers and get the self-confidence to write and publish as it can be a scary process and you don't want to make any dummy mistakes with the process when you have an actually important paper at hand. Like car driving, there is a reason you drive in driving school with your instructor. It's pointless drives to nowhere in the middle of actual real traffic, but they help you to master the skill and drive to work later. Dec 31 '20 at 21:37
  • 1
    @Frank: I think we must maintain the distinction between a pointless paper and a paper of modest value. I encourage my PhD students to write papers before their PhD thesis in order to train them in this way. But I would not encourage a student to write a paper that contained no new results. Jan 3 at 19:59
10

One obvious reason is the "information asymmetry".

People indirectly paying for papers (or giving jobs to people with 123 published papers) do not understand enough about the field to properly rate them.

So if you have no morals, the best way to increase your income/status is to pump out garbage papers... remember to put IoT, crypto, and deep learning in the title!

You can see this not just in academia, but also in companies where poorly run companies hire people based on their CVs (e.g. bunch of useless certificates or membership in some fancy organization)

3
  • 2
    I strongly agree.
    – Trunk
    Dec 29 '20 at 14:17
  • 11
    I know of a country where they added a law that if a prisoner publishes a paper (or completes a degree, or publishes a book), there will be a certain reduction in the sentence. Sounds like a noble idea to promote studying, right? Until it turned out that some imprisoned and barely literate mob bosses started publishing dozens of papers per month...
    – vsz
    Dec 29 '20 at 17:31
  • 1
    "IoT" is well into the hype cycle (already in trough of disillusionment?). The new buzzword is fog computing. Dec 29 '20 at 17:46
10

In addition to the excellent answers above, let me add another reason: project requirements.

When we submit a research grant proposal, we often have to indicate how many papers the project is going to produce if funded. This is in particular a typical requirement for nationally funded projects in some countries, less so for EU or Western academia funded projects, which do tend to offer more flexibility to researchers. Similarly, publishing in Open Access journals in particular is a mandatory requirement in some projects.

When such a project ends, the project's team is responsible for having delivered at least the promised number of papers. Not only the reputation of the institution is at stake, there might be "financial corrections" - i.e. the funding institution may ask to pay back some of the grant money, if the project has failed to deliver all that it has promised.

These results are evaluated by administrators, who are not actually reading the papers, but just checking whether they have actually been published. Furthermore, the evaluators often do not care if there were extenuating circumstances such as a researcher taking a parental leave during the project, a researcher moving to the industry, or (I imagine) even a researcher actually dying.

This leads to heavy pressure from ones institution to publish some minimum amount of papers every year.

7

Publishing peer-reviewed papers has replaced answering research questions as the main goal of the research enterprise, and scientists are pressured to publish even when they don't have anything interesting to say. Most research projects start and are brought to conclusion under the assumption that they will result in multiple peer-reviewed publications, regardless of whether the study is successful in answering the research question or not. This is because of the ubiquitous pressure for publishing in large quantities institutionalised by the "publish or perish" culture in academia.

Quantity over quality is the name of the game in many circles, mainly because of incentive structures created by funding allocation and hiring practices. Bureaucrats working at university and funding organisations, as well as researchers in multidisciplinary review panels , lack the field-specific and topic-specific knowledges to judge the quality of one's work, so they need to rely on a proxy. In most academic settings, publication in peer-reviewed, indexed outlets is what is used as that proxy. Because there are enough indexed peer-reviewed journals that virtually any paper can get published somewhere given enough persistence, it becomes a numbers game. People in those funding and hiring committees don't have time, resources, and skills to read through all papers in someone's list of publications, so they just resort to counting them. Sometimes journal and conference prestige is taken into account, but more often than not it is just the publication count that matters.

Given the above, researchers operate under an incentive structure that leads them to taking any opportunity to get stuff published and to never let a project go without publishing (regardless of how unsuccessful or unimportant the study is). If it can be counted as a publication on somebody's CV, it will be written up and published.

2
  • 2
    So many people in academia share this concern. Yet we see very little public debate on this within any university or even in the public arena . . .
    – Trunk
    Dec 30 '20 at 18:23
  • 1
    Very well put. (Not to mention being sadly-true...) Dec 30 '20 at 23:28
3

The other answers here are excellent, and I won't bother repeating them.

One more obscure possibility is that some of these are "defensive publications." A company can develop a proprietary technique and not consider it worth patenting, but are concerned that a competitor may patent it and deny it to them. If disclosed in a publicly available paper, it is now in the public domain and any subsequent patent can be invalidated legally.

1
  • Yes, that can happen. Another fringe possibility is April fools, and papers made to demonstrate the lack of proper review. Sadly, many of the downright wrong and pointless papers I see in crypto are neither; the only thing they show is ignorance of basic principles of that science. My heart bleeds when I see them studied, perhaps mandatorily.
    – fgrieu
    Jan 5 at 5:48
2

Just a footnote rather than an answer. I have worked in crypto-related fields and from what I have seen there is a lot of 'security by obscurity.' People implement algorithms, write about them, study them and generally make a lot of pointless noise. Getting to some real info in amongst this low signal to noise ratio is hard. I don't know if it's part of the game, or if it's just the crypto-currency industry with its obsession with 'decentralisation' causing a lot of disorganised redundancy. Probably a bit of both.

5
  • 1
    What are they trying to secure?
    – Ambicion
    Dec 30 '20 at 14:57
  • @Ambicion Both jobs and the weaknesses in protocol implementations.
    – Frank
    Dec 31 '20 at 9:18
  • 1
    Fundamentally it's protectionism. Protecting their jobs by publishing stuff that promotes their own "brilliance" yet without enlightening anyone capable of equaling them. Protecting their spare time for going to exotic conferences and playing leisurely with their next golden idea in their comfy offices.
    – Trunk
    Dec 31 '20 at 16:59
  • 1
    @Trunk I am a mathematician, and I suspect such protectionism is exactly what happens in many branches of math. There are entire publication mafias based on keeping a level of obscurity on their topics. Sadly, also on some of their theoretical flaws.
    – Ambicion
    Jan 2 at 16:35
  • @Trunk Nail. Hammer. Head.
    – Frank
    Jan 4 at 12:01
-1

Publication worthiness is a condition of most funded academic research work. This results in a lot of papers each year on every field. There is nothing wrong with that. But maybe we need to categorize our journals better as regards the seminality of the work presented. This is not to say that papers particularizing or generalizing existing findings, those showing anomalies to a general trend or simply modelling/remodelling a well-understood phenomenon should not have a place in journals. Clearly they should. But probably not in the same journals as seminal work. One might consider other categorizations also, e.g. on applicability of research work.

On your point about specious or doubtful papers, this reflects on editorship and more pointedly on readers control over editors. Maybe it would be healthy to have subscribers vote each fall to retain or dismiss existing editors. Better still, there might be a maximum term for each editor and due care to ensure new editors are not simply proxies of the old ones, e.g. ex-students, ex-colleagues. I accept that for specialist journals that were founded by a small number of dominant researchers and which has a limited number of subscribers this would not be easily done.

The futile study of pointless papers is often due to many references to them in review papers or textbooks. It may be ameliorated through a rating system. The internet allows users of any product or service to rate it. Likewise serious critics (credentials provided, of course) of a paper might be allowed to rate and re-rate it in terms of content validity and current relevance. Anyone preparing a review paper would then be wise to have regard to its rating before referencing it.

Finally, I think all stakeholders in research (and not just active researchers) should speak up more on occasions like those mentioned above. Usually we content ourself with shooting off to a sympathetic colleague in the privacy of our office - then neglect to do anything more about it. This results in the next crop of researchers falling into the same frustrating hole as we did.

Professionalism includes a duty to challenge the competence of our colleagues (as well as that of ourselves) when it is lacking.

4
  • 1
    Individual subscribers are not relevant to the typical academic journal. This answer appears to be unaware of peer review. Jan 1 at 10:16
  • None of the suggestions above are exercised - or even discussed to my knowledge - at the present time. I take the point that "subscribers" includes a lot of university libraries with no immediate competence but who act on direction from the relevant department to stock a given journal. I don't want to add unduly to academics' workload (!) but it would not be amiss of faculty to question a journal's editorship where doubtful papers are published. I'd have more regard to your criticisms of my proposals if I knew your own views on how to deal with these matters.
    – Trunk
    Jan 1 at 13:11
  • Peer review is how stakeholders speak up. It is not perfect, but the failings are more often failures of the peer reviewers than the of the editors. Jan 2 at 2:05
  • Just looking at review models here elsevier.com/reviewers/what-is-peer-review shows the moral hazards arising. (Please advise as to what your favored type of review is.) I and my contemporaries were never told a damn thing on peer-review when starting a PhD. Maybe you had the same experience. All guys I knew uniformly followed supervisors' advice on publishing - maybe even left all journal contact to them. One thing that should be known when submitting papers is what type (in lewd detail, please) of process is employed by the journal. I doubt if this is so in major journals.
    – Trunk
    Jan 3 at 13:16

Not the answer you're looking for? Browse other questions tagged or ask your own question.