9

Why do sites that host papers such as IEEE's not have a comments section for each paper? Sometimes there is something that needs to be added, corrected or clarified by the community but as far as I know there is no such a thing... or is there?

Improve this question
10
  • 8
    What could go wrong? First, what do you mean by community - anybody could add a comment? I'm sure nobody on the internet would get kicks out of defacing/trolling/harassing yet another website...
    – Jon Custer
    Commented May 19, 2017 at 0:50
  • 5
    @JonCuster On this site everybody can comment and it's not that bad.
    – Federico Poloni
    Commented May 19, 2017 at 6:27
  • 1
    @FedericoPoloni Actually you need at least some amount of reputation (like 20 or 50) to comment. I don't recall if the association bonus counts towards this, but it does mean that someone completely unfamiliar with the SE network cannot comment.
    – Tobias Kildetoft
    Commented May 19, 2017 at 10:34
  • 2
    @FedericoPoloni... On this site, if we did not have myriads of users flagging spam and abusive posts, it would soon be as useless as most other sites.
    – GEdgar
    Commented May 19, 2017 at 11:04
  • 1
    @JonCuster Not anybody, registered users usually have paid actual money to get access which cuts a big portion of random people logging in and writing trash. Also, researchers have critical thought and better judgment to decide what is important and what not.
    – Anna K.
    Commented May 19, 2017 at 17:20

3 Answers 3

Reset to default
6

Reason 1: Many publishers (still) see themselves simply as content providers. A reader goes to the publisher website, downloads a pdf and leaves. A publication is understood as a static (dead?) document, so post-publication activities are not encouraged technically.

Reason 2: The publisher website landscape is very fragmented. Even smaller scientific communities have to go to multiple publisher websites to get their content. So, a researcher would need multiple logins to add comments on the different websites. A high barrier even today.

Reason 3: A simple forum-like comment covers only a few of the post-publication discussion use cases. It is good for general opinions and corrections. Annotations, however, are much more targeted and also cover use cases like corrections of specific mistakes in formulas, paragraph-level questions and statements, discussions around data and images, proofreading, and referencing of specific parts of a document (example of direct linking at PaperHive: https://paperhive.org/documents/jKRG3MqMuj66/revisions/QOdeW6lxZMHk?a=s:Vjw91UM7br1A).

These are some of the reasons why we at PaperHive [Disclaimer: I'm one of PaperHive's founders] chose to build a system that is connected to publishers' websites but enables the communication on a separate cross-publisher platform entirely dedicated to reading and discussing. We have a small widget that informs readers on a publisher website of existing discussions (see example on a partner's website: https://www.growkudos.com/publications/10.1016%252Fj.neurobiolaging.2014.04.026) but the actual reading & annotation happens on PaperHive.

All public discussions are under the CC BY license and we follow the W3C open annotation standard (which is the foundation for interoperability and portability). We have an API for the export of annotations (documentation in progress) and have on our roadmap to archive the discussions at one of the two most-widely used systems in publishing (Portico or Clockss).

Improve this answer
7
  • 1
    Thanks for chiming in! Seeing that there is a W3C annotation standard and that someone uses it makes me happy, and CC BY looks very good. Though I'm wondering why I often get an infinite loading screen when I click on the "referenced text" in a discussion. If you are pulling these documents from the publisher page in real time, you should probably build safeguards for the case the publisher page is down or changes its structure or the document gets re-scanned and the references lose their target.
    – darij grinberg
    Commented May 19, 2017 at 15:47
  • 1
    On a deeper level, though, let me describe what to me is the most uncanny part of your site: It all relies heavily on an internet connection. Unless I have missed it, I cannot find an "download the whole discussion for a given paper as a PDF" (e.g., concatenate the paper with the discussion threads, and replace the yellow highlighter by some sort of endnote markers) option. But that becomes useful, once discussions become long and meaningful and begin containing their own research. Perhaps StackPrinter could be a good model here.
    – darij grinberg
    Commented May 19, 2017 at 15:51
  • 2
    Either way, what you've built there looks like a great step forward! I'm looking forward to eventually referee papers on such a platform...
    – darij grinberg
    Commented May 19, 2017 at 15:52
  • 1
    Minor thing: Is there a way to search for a given subject and sort by "how much discussion has been recently made about the paper"? That could significantly improve adoption, as it would automatically provide people with examples of usage relevant to their discipline (once such examples exist).
    – darij grinberg
    Commented May 19, 2017 at 15:54
  • 1
    Thank you for the comments, @darijgrinberg! - You have a point with the dependance on publisher's tech. However, changes in publishing are extremely slow. Html and text-matching will be one way to secure portability of comments. - "Download annotations" is on our roadmap. Thanks for the StackPrinter hint! - There are already researchers conducting peer review on arXiv papers within PaperHive. They review the papers privately and send the annotated documents to the editors of some journals. The open access publisher LangSci Press also has their proofreading and peer review on PaperHive
    – Alexander Naydenov
    Commented May 23, 2017 at 8:26
7

There are a few third-party sites where you can do this. For instance, https://paperhive.org/ and https://scirate.com/. None of them has a large user base, at the moment. There used to be more (http://www.arxaliv.org/, https://selectedpapers.net/) but they seem to be down.

(Disclaimer: I am a friend and a former colleague of one of the founders of Paperhive.)

Improve this answer
4
  • Does PaperHive support LaTeX/MathJax? How easy is it to export all discussions related to a given paper from PaperHive? (Or even view them all together, without having to click around a PDF.) What are the plans for long-term archival and availability? What is the copyright policy on comments?
    – darij grinberg
    Commented May 19, 2017 at 8:29
  • 1
    @darijgrinberg I can answer only some of these questions: Paperhive supports Mathjax in comments; the comments are CC-BY licensed. For the other ones, I will point out this thread to the Paperhive people and hopefully they'll answer themselves.
    – Federico Poloni
    Commented May 19, 2017 at 8:39
  • Thanks for this! Such a site would be very useful, but I am wary of something that will fold in 5 years and leave behind a lot of non-migrable (for reasons of format or of licensing) resources. So far, many of the pioneering plants have withered and little has been left behind; if I hadn't archived my arXaliv comments, I'd have no idea where to find them now...
    – darij grinberg
    Commented May 19, 2017 at 9:06
  • @darijgrinberg Answers by a Paperhive founder are in this other answer.
    – Federico Poloni
    Commented May 19, 2017 at 12:37
2

There are a couple issues with this that I've both seen and experienced:

  • It's probably not worth it, as few papers actually draw comments. Pubmed Commons, for example, means you can leave comments about a paper on it's Pubmed entry. I looked up some of my papers...there isn't a single comment. Not one. And while I'm not the most prominent researcher in the world, we are talking about a body of papers with 500+ citations. So...why bother with implementing this?
  • If you do get comments, you're vulnerable to things. For example, the anti-vaccination community launching a spam campaign. While you can restrict the comments to subscribers, what about open access journals? Crackpots with library memberships? Many of the same arguments that you should be able to comment apply to the public being able to comment. And that means moderation. Whose going to do that?
  • What are the obligations to commenters? Am I free to ignore them? Or, if J. Random asserts there's an error in my paper, do I have to address it. How much do I have to address it?

TL;DR: There's significant costs and not a lot of clear benefit, especially when there are ways to contact an author directly.

Improve this answer
6
  • But that's the first reason I wanted to stress this issue: some papers might have errors, the rest of the world shouldn't suffer from those errors... or even worse... propagate the erroneous information onto another paper or use them to his/her advantage. If people notice something it should be pointed out and not hidden just because the author will have xyz problems.
    – Anna K.
    Commented May 19, 2017 at 20:28
  • @AnnaK. One of the problems though is that "Commenter Thinks There's an Error" and "There Is an Error" are not the same thing. What does the journal do if the author refuses to comment? Or the commenter goes on a string of nebulously goal-posting objections (see: every anti-vaccination argument ever). Or the author and the commenter disagree that the error is in fact an error? How long am I obligated to respond if indeed I need to respond at all? If there's an important error, that's what letters - either to the editor or the journal, are for.
    – Fomite
    Commented May 19, 2017 at 20:31
  • Most of these problems are trivial (in my opinion). 1: Since these are comments, the author should not feel obligated to answer them. This is why conferences are: the real obligation to answer questions is there. The comment section should be something that people can comment and users to be able to see for years later. 2: Just a report button will do for people that disrespect the comment section, papers are meant to be read by grown ups not 5 yrs olds. 3: I'm sure journal and conference editors will be happy to receive tens of emails everyday for things posted decades ago (possibly).
    – Anna K.
    Commented May 19, 2017 at 20:45
  • 1
    @AnnaK. If I'm free to ignore it then whether or not there's an error is never really resolved. You've just added noise. As for the "report button", a report button needs someone on the other end to make the judgement over whether or not that's actionable. And again, if you just want to muse about a paper, there's already a mechanism for that in many journals.
    – Fomite
    Commented May 19, 2017 at 20:47
  • 1
    Better do noise than let things slip silently.
    – Anna K.
    Commented May 19, 2017 at 20:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .