Public accountability for academic work

Question

I have realised that my performance improves when I make my progress publicly accountable. For example, my jogging times improve and I lose more weight when I enter myself onto a public rankings list. I would imagine that my academic progress would also improve if I make my progress publicly visible.

I'm wondering what online rankings or any other publicly visible progress reports might be available in academia? I've heard of impact factors, but is there anything else? I'm sure there are people against turning academia into a competition, but I find some sort of competitive element really improves my performance.

Answer 1

You can always compare the length of publication lists, e.g., using Google Scholar, ResearchGate or even your colleagues' homepages.

Summaries of publications and their impact are (attempted to be) provided by the h-index and similar measures. (Impact factors really pertain to journals, not researchers... but indices for researchers may take into account whether the researcher publishes in a low- or high-impact journal.)

There can be, uh, lively discussions about the merits or not of such measures, especially in the context of hiring and remuneration decisions.

One problem with using such measures for short- and medium-term motivation as you propose is the lag between the work and the publication - feedback is far too slow. If you are looking for motivation, you may want to check Productivity.SE.com.

Answer 2

Some more ideas to complement StephanKolassa's answer.

Another possible idea that crossed my mind multiple times in the past is making a Trello board showing my papers in different columns according to their progress status, with a scale such as:

• 0: preliminary idea, not clear if it will even work
• 1: some experiments or theoretical work done, seems promising; will likely lead to a publication but lots of work needed
• 2: most of the experiments/proofs ready, still to write down in coherent form
• 3 draft mainly ready
• 4 published as preprint and submitted
• 5 first round of positive reviews, awaits modifications or already resubmitted
• 6 published; congratulations!

Other random ideas: if you are writing a long document such as a thesis or a book, you may want to publish online a graph tracking the number of pages written vs. time. Similarly with lines of code. If you are using git you can gather lots of statistics using GitStats (example)

Answer 3

Microsoft Academic Search maintains top-something lists of researchers in different fields (e.g., top-100 researchers in software engineering, top-20 researchers in WWW in the last 5 years, etc.). Clearly, you will only start showing up in these rankings after you already had some number of cited publications, so it is probably not useful for your PhD research. Also, as I have experienced myself, the ranking is weird, so I wouldn't say it is exactly a reliable way to measure your progress. Finally, it should be noted that the feedback achieved by these rankings is going to be very slow. In fact, you will (theoretically) see improvements in these rankings years after you wrote high-impact papers. It is not exactly that you can use it to track how good your last week was.

You could also use Google Scholar and compare your profile against profiles of other people of similar academic age or standing. The good thing here is that you should at least show up as soon as you have co-authored something. However, similar restrictions to above apply - progress will be very slow, and any comparison based on e.g., Google Scholar h-index will be a noisy heuristic at best.

Answer 4

As many answers already pointed out, academic progress is slow. Changes in comprehensive rankings will take their time. I assume this is the reason why people say that a person in academia must endure frustration and have tons of patience. On the other hand, humans seem to usually need a much more frequent feedback in order to actually motivate yourself.

I find one little trick particularly helpful: find colleagues that work on closely related topics. Share information and work together on the same project. Seeing someone else having success on his or her share of the work will surely make you want to contribute as well (if not more). I must admit, that this will only work, if everyone involved aims toward the success of the project. There are various ways of measuring, publishing, and rewarding progress in this case: the tool can be as simple as a progress bar on an intranet's webpage that steps along as milestones are reached, money for visiting a conference on the condition of a publication, or the already mentioned Trello board that just visualizes how the contributions of different people in your group help a project advance.

Sharing information in a small study or work group breaks down the long-term progress into the small steps that academia actually enforces: idea, thinking, trial, and finally success or failure. Adding discussion at any state of this progress will help to uncover misunderstandings and sure failures faster than if you would work on your own. Linearizing your thoughts to actually tell them so someone enforces a clear picture in your head, again ruling out possibly overlooked details.

Furthermore, swapping ideas will help everyone in your group gain insight and see the same world with different eyes. To every topic (how old and worn out it may be) there is a perspective or context that has not been considered yet. Finding out about these is something I found particularly motivating during my time of study. Keeping the discussion with colleagues and friends alive will surely help you with this.

My approach to answer your question is: choose a method of feedback that suits you and try to shorten the intervals. The method of feedback does not matter as much as the time you have to spend without.

Lastly, a small excursus: We have some source code for numerical computations that we share among the PhDs in our work group. It is always a bit frustrating to find oneself spending hours of debugging someone else's mess.. Recently, we started offering bounties (you know those coconut-chocolate-bars?) for bugs someone would find in someone else's source code. Now one has one more reason to watch out bugs on your own (to avoid spending a "bounty"). Moreover, the frustration is somewhat mitigated in case a bug is found since one receives a (small) compensation.