4

We are conducting a simulation study to compare various clustering methods. However, the implementation of one of the methods used in the study (Mclust) occasionally produces empty clusters, which is not allowed in our study. We only use R in the study. Unfortunately, reimplementing the methods is not a feasible option. However, we don't want to exclude this method from the study.

How should we mention that "we know it occasionally does not work the way we want, but we still want to include it in this section"?

Improve this question
4
  • 35
    Just a friendly piece of unsolicited advice, but I'd recommend being very sure that the "software error" is not just "user error".
    – Ian
    Nov 1 at 13:40
  • 12
    This doesn't sound like an error so much as you using the software in an unexpected way.
    – Teepeemm
    Nov 1 at 23:27
  • 3
    It looks like this may be a known issue, and (if I have understood correctly) is related to the fact that Mclust does not produce clusters in a deterministic way: stackoverflow.com/questions/64323572/…. If that interpretation is right, then I think it's valid to explain why this issue arises (non-deterministic clustering) and the method you chose to deal with it (only keep realizations where all clusters are non empty).
    – Andrew
    Nov 2 at 1:44
  • 7
    Incidentally, if my above comment is the correct explanation for your issue, then your issue is not a software error but rather a subtlety in interpreting the results of the method, which would be present in any correct implementation.
    – Andrew
    Nov 2 at 1:46

2 Answers 2

Reset to default
12

Ideal solution: you report the bug (you say it's a software error and not intended behavior) to the developer, they fix it promptly, and you obtain your results with the new version.

If this doesn't work out, you can mention the exclusion criteria for the cases you don't use in your analysis as is detailed in EarlGrey's answer. In principle, this is no different than other methodological limitations of your study: reviewers may find it acceptable or objectionable, or help you mitigate the limitation. Regardless, it needs to be disclosed. If possible, estimate the potential impact on your conclusions.

Improve this answer
10

Are you comparing the methods or the implementations of methods? Do not mix them.

If the method behind "Mclust" is what you want, then the implementation is "Z". Use it only when its results match the assumptions of your study (i.e. only when it does not produce empty clusters).

In the text, you should describe the limitation of the implementation (i.e. "Z results are used only for the cases that ...".).

If you cannot find another implementation of the same method, maybe you are lucky and during peer-review you may receive the revision "use "ZY", it works for the cases of Z and also the cases with empy clusters".

Improve this answer
2
  • 1
    Thanks for your suggestion. I edited the post.
    – Kelvin Grove
    Nov 1 at 10:15
  • 4
    People often append modifications to the name to make it clear. If the original method is "Foo" you call it "Foo-NoEmpty".
    – user71659
    Nov 1 at 19:15

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .