What to do when reviewer wants p-values, but I prefer to use significance stars to prevent clutter in table?

Question

My regression tables have coefficient estimates, standard errors, and significance stars indicating the range of p-values (e.g. two stars means p < 0.05).

A reviewer wants the exact p-values. I personally think it would make the table more cluttered than it already is, and I have 12 regression tables.

Should I adopt the reviewer's suggestion, simply because they are the ones who will play a big role in determining whether my paper is accepted? Or should I attempt to explain, saying that the p-value can be calculated from the coefficient estimates and standard errors (besides, the significance stars already give a good guide to the p-value?)

=====

Edit: I decided to get rid of the significance stars and insert the p-values. Thanks for the feedback! (Interestingly, the paper was accepted, and the reviewer recommended the paper for acceptance, but they suggested I have both the significance stars and the p-value. But I didn't take their comment)

Answer 1

In general you are not obliged to comply with all reviewer wishes in your revision, unless maybe the editor has mentioned such a point explicitly in the decision letter. If you don't want to follow a reviewer's suggestion for revision, just explain in your response letter in a factual way why you choose to do so, and see what the editor and reviewer think about it. You have good points for such a response in your question already.

I would nevertheless suggest to make the reviewer's life as easy as possible. Even if you don't want to have them in the paper, you could maybe add a list of p-values to the response letter for exactly that comment, stating that for the convenience of the reviewer you calculated the p-values from the data given in the paper (instead of expecting the reviewer to do this). You could also offer to put such a list in any supplementary material if the journal has such and the reviewer thinks it's important that readers see that.

Answer 2

Something like this should not influence the final decision of the editor. Further, this is not a fight worth having with the editor/reviewer so you should be flexible and accommodating.

You should check past issues and see what the format is. If p-values, regression coefficients, and standard errors are typically provided, you should probably do that. If only 2 are provided, go with those. Assuming regression coefficients and standard errors are typically provided, in your rebuttal letter a statement along the lines of

Following what we think is the standard format for Journal Title, we have only included the regression coefficients and standard errors. As the p-value can be estimated from these values, there is no loss of information. We would be happy to make a change if our understanding of the standard format is incorrect or if an exception should be made in this case. For your convenience we have attached a copy of the tables with the p-values included to this letter.

Answer 3

Subject Matter Answer: You should do as the reviewer asks. Regression stars are vague and non-specific, and amplify differences that are actually trivial. They are a coarser version of the same information, and considerably less useful.

I'm also skeptical that the difference between "*" and "0.46" is what's going to make your table too cluttered.

If anything, the most defensible answer based on "the p-value can be calculated from the coefficient estimates and standard errors" is to drop both.

Academia Answer The answer to what you should do will depend very much on the reviewers, and how big a deal they want to make of this - as well as the journal's stance on it. There are journals I review for where this objection (and indeed, the p-values generally) are likely enough to derail the entire paper. There are others where this is of less concern.

The question then becomes "Is another round of review worth potentially not having to reformat my tables?"

I can't imagine a circumstance where I would answer "Yes" to this, but YMMV.

Answer 4

In addition to the suggestions to include the exact p-values to make the referee happy, I also want to add that including the p-values explicitly (rather than making your readers calculate it themselves if they want to know) will make your readers more happy as well.

When I read a paper, I hope the author has included as much information as I need to understand what I need to from the paper without having to do a lot of footwork myself.

Answer 5

Explain that the table would be too cluttered and include a full table with the p values in the response.

Sometimes requests like these are there to determine if the values (p values in this case) are acceptable in all cases. "I want to see the values" does not always mean that they must be in the paper. He/she might literally mean that he/she wants to personally see the values for review purpose.

As a reviewer, I have asked extra info just to see if everything is alright and as an author I have provided reviewers with extra information on the reply to assure that everything is alright and missing things are really not there only because it really does not fit into the page limits.

Answer 6

I understand your objection to putting in redundant information, but are your tables as-is really all that cluttered? With just coefficients, standard error, and some coded asterisks, along with a label for each row, that sounds like just 3 columns (no need for the asterisks to be their own column).

Unless I am missing something, I'd recommend just adding the p-values to appease the reviewer in your revision. If it causes a problem with typesetting and organization, you could ask the editor for their preference, which might overrule the reviewer or result in a movement to supplementary material.

Just as a personal preference, I'd usually rather see coefficients and confidence intervals rather than standard errors, p-values, and significance stars, if we are limiting to just a couple items. Confidence intervals are much more flexible.