How to improve the quality and visibility of research?

Question

This might sound like quite a basic question, but the answer has eluded me for the past couple of years. I'm from a computer science background, but the question is probably relevant outside this discipline.

Please forgive if my register for the rest of this question is quite frank. I'm coming to academia stack exchange instead of colleagues in order to talk about this matter candidly. I'm also sorry if the question is a bit long-winded, but I think it might be necessary to get past the stock answers.

The edict of "publish or perish" is a standard rejoinder, and beyond that the onus is for works published to be highly visibility, and of good quality. What this actually means is debatable though.

Most research cannot be reproduced, provides no code, and in practical terms is only useful for the purposes of citing (e.g. X et al. tried method B and got results F, but we don't have access to that data set, they don't say if they tried other, simpler methods, and what results they got, if they did indeed try these methods). Very often complexity seems to be pursued for the sake of complexity, whereby approaches are obfuscated with roughly annotated formulae written in Greek, standard methodologies that everyone knows are superfluously provided (and explained badly), and really convoluted approaches are adopted and are to taken as so, by the reader (an example would be a Bi-directional 5 layer CNN-LSTM-CNN-LSTM-LSTM for an NLP task with an explanation of "this obtained the best results"). Actually useful research material... blog posts, how-to guides, stack overflow answers, discussion group records, are all of zero academic merit.

I find myself trying to ape the aforementioned papers, but I am struggling to make them complex enough (certainly from a visual point of view). So the content itself is a challenge, because content has to be original, and the main way people appear to be guaranteeing their originality is by heaping one complex method upon another. Maybe if you are part of a big team this is feasible, but I'm just one person essentially working by myself.

Which brings me to the second aspect: visibility. There is such a baffling array of conferences and journals. Some of them are bogus, and one can filter those out. There are a couple of "top" conferences that everyone flocks to, which consequently makes acceptance a crap-shoot (if they accept 200, and 3,000 apply, you are likely going to have a lot of "good" material not make the cut simply due to the numbers game). All the rest (hundreds, if not thousands of peer reviewed conferences etc.) are really difficult to measure. Look up the ranking of a conference on one site and it's B1, and on another site, it's C. There is not only an opportunity cost here, but a temporal one - having anything submitted and accepted takes several months, and making the wrong decision can be costly on both fronts.

So the standard advise of "publish good work through peer-reviewed avenues" is only half the answer, but I'm unsure how to make my papers more liable to be the ones that are cited.

Sorry if this post comes off as cynical, it's simply trying to be both realistic and practical. I feel as if there's an approach that should be obvious to me to pursue, but at the moment it seems that I am missing out on the vital ingredient to turn from meh to Michelin.

Answer 1

How to improve the quality and visibility of research?

I think this is a very relevant question, but also a very broad one. Clearly there is no simple answer, so I will try to give a few general directions. Also I think it makes sense to distinguish between the field-specific aspects and the more general aspects across academic fields.

Reproducibility in Natural Language Processing (NLP)

I happen to work in this field as well and I can relate to the problems described by OP. The field has a quite serious problem with reproducibility: this is mostly due to NLP being a very experimental domain, where experiments often involve a lot of complex parameters and where data is extremely diverse by nature. Additionally there's no strong incentive for strict reproducibility protocols in NLP, as opposed to other domains where mistakes are not an option (e.g. health or security).

Still, the community is aware of the issue and has been trying to address it for some time:

• There's now a well established tradition of "shared tasks", scientific competitions meant to (1) provide benchmark datasets and (2) compare different approaches in a uniform way.
• While still not the general rule, it's becoming the norm to not only publish a paper but also the code and/or data used for the experiment. Conferences have been encouraging this for a few years now by allowing the submission of supplementary material.
• Through various other initiatives: proposing a specific category for reproducibility papers, building a repository of language resources, etc.

Of course technical results ("new model X improves performance by Y%") are the bread and butter of conferences, but my impression is that the best papers acknowledged by the community (e.g. best papers awards at conferences) are rarely pure technical papers, original ideas and insights tend to be valued more (but this might be a subjective perception based on my limited experience).

Visibility and academic reputation

blog posts, how-to guides, stack overflow answers, discussion group records, are all of zero academic merit.

This is not completely true, because:

• Building a reputation through various means can help your research get cited
• Outreach and communication towards non-academic publics is increasingly encouraged (see for instance the growing trend of citizen science projects)

However it's true that these activities (unless very successful) are usually less valued than the good old publication record. Unfortunately at many levels research and researchers are evaluated against simplified indicator (publications, citations, h-index...), despite the obvious biases in this. Note that most researchers are aware that publications are not necessarily synonymous for research quality, but funding bodies are happy to use this kind of performance indicators.

My personal publication strategy is along the following lines:

• Target top conferences/journals for any (potentially) significant contribution.
• Target reputable specialized workshops or small conferences for prospective ideas, work in progress and other kinds of moderately significant contributions. It adds a modest line to my publication record, but more importantly this way I can meet and see what the community is doing on a specific topic.

Caution: I don't have a particularly successful academic career myself, so don't assume that my advice is good ;)

Answer 2

One way to enhance your visibility and reputation is to form deep and lasting collaborations with as many people as you can. Explore research ideas with them. Share with them. Write joint publications with them.

This is not only a source of ideas, possibly more important ones than you can come up with yourself, but as their reputation increases so will yours. If they tend to get cited, your joint publications are cited.

The following is intended to be a bit humorous, not egotistical. Sunday Funnies, so to speak.

I learned recently that my Erdős Number is three. That sounds impressive until you think about what it means. All it means, necessarily, is that I've written at least one joint paper in my life and my co-author(s) have been active writing joint papers and eventually one of them wrote a paper with a person at Erdős-1, making them a "two" and me a "three". That is as low as I can get (with a traditional definition) unless I actually do some work, since Paul Erdős is no longer alive. I could in theory become a "two", but to do that I'd have to write a joint paper with a "one". I actually know such a person, maybe several, but the likelihood of actually doing the work is small.

If you use a somewhat expanded definition of Bacon Number, I could conceivably be a two. Normally Bacon number is defined in terms of film acting. The only film I was in I was the sole actor, but I "acted" on the stage once in my life. But I did so with a few people who were more active in that world. So, if any of them has a finite Bacon number then so do I, provided you accept stage acting, not just film. If any of them has actually acted with Kevin Bacon, then I'd be a "two" without any additional effort on my part. But note that all of the work is theirs, not mine. Just as for the Erdős number. But note that if I were to do a lot of acting (with others) my skill would likely improve, not just my reputation. That is really the main point of this humorous aside.

Write a lot of joint papers, sit back and collect reputation.

Of course, writing a lot of single author papers is also likely to increase your reputation, but you don't, then, get the benefit of sharing ideas and the synergy that can come from that.

---

Hmmm. Full disclosure. My joint professional work is actually in computer science. Many, but not all, mathematicians would accept that as "close enough" to contribute to my Erdős standing. For those who won't my Erdős is infinite.