It is widely considered that there exists a publication bias whereby it is more difficult to get null results published than it is to get positive results published in peer-reviewed journals. As a result, we might expect it to be more difficult to publish the results of a study which failed to find evidence in support of a hypothesis than one which did.
In many of the health and social sciences, an important area of work involves evaluating assessments (e.g., of mental abilities/traits, physical ability, properties of a social context, etc.) to establish that they are reliable and valid measures of whatever we intend to assess. In at least quantitative studies, this typically involves evaluating the assessment against a bar (say, that a correlation coefficient measuring inter-rater reliability should be >.7), and if it passes this bar then we take this as evidence of the reliability or validity of the assessment.
What I'm wondering is whether there is a publication bias here as well? If I evaluate an assessment and fail to find evidence that it's reliable or valid (say, the correlation was .2), am I likely to find it harder to get it published, much in the same way as I might expect to if I was to fail to find evidence of some hypothesised effect?
In particular, I would be interested to know whether anyone has had experiences of such a publication bias, or knows of any writing on the topic.
It seems very rare to find studies which report on how poor an assessment is, but obviously I can't rule out the possibility of another publication bias being at play here (such as the file-drawer effect).
