errors in R code rarely cause the session itself to crash; usually they result in an error message, which one may easily find the source of
yes. There are a few situations where session crashes are more probable:
- with compiled code and
- libraries installed on the system (as opposed to R packages) which are incompatible across versions/OS boundaries
- particularly libraries that are "close" to the OS (I've had trouble in the past with interactive graphics across OS boundaries Linux vs. MacOS vs. Windows)
Thus, if the lines in question can (easily) be replaced by equivalent (though maybe slower or more memory conuming) R code, the crash may be avoided and the scientific merits of the model may be evaluated.
We may have an example here of the famous "premature optimization is the root of all evil".
Amount of cooperation to be expected?
As author and maintainer of scientific/academic software packages I certainly appreciate bug reports etc. I even more appreciate bug fixes, btw.
However, I also have only limited resources available to maintain such software, and the longer back the publication was, typically the less resources (e.g. new employer is not willing for their staff to maintain projects at former universities) are standing agaist increased maintenance requirements. (Probably less of a problem here as the paper is quite recent compared to some code that was developed years and years back)
Please also have a look at Why do many talented scientists write horrible software?
The authors may be seeing the publication of the code as them being extremely nice in giving the public not only the abstract description of the model but even an example of an implementation.
In fact, they may even have had an uphill fight to get permission to publish their code under an open license (I've been in the situation of not being permitted to do this by an academic research institution.)
Of course it would be much nicer if they had written more rugged code in the first place or would now help you debug. But you don't have any kind of hard right to their time.
I'm chemist and we sometimes have lab procedures that are difficult to describe in such detail that another lab with somewhat different conditions is [easily] able to reproduce them. Thus, people visit other labs to learn their techniques - this means a whole lot of effort for both sides and IMHO that needs to be appreciated. Similarly, I appreciate if a package maintainer does put in the time to deal with my bug report. And in consequence, I try to make it as easy as possible for them.
There may be a mismatch in software development ability/possibilities here: guessing from the crashes that their code isn't at the highest level of robustness and that you didn't find them using continuous integration etc. vs. you providing them with a Singularity container: they may not know how to use that or may not have the possibility to use it or may not be willing to put in the time to get that particular virtual machine up and running.
I may add that in many places (including universities and research institutes) there is in addition daunting burocracy to get the IT department to install further software on their machines (and they are the only ones with installation rights). I teach for the carpentries and it is not unusual to find course participants do not have the necessary software for this reason - even if they are officially sent to the course.
You may get further if you ask them whether they do have the ressources to resolve the issue with your help and how you can help resolving the issue.
I do not feel that this amounts to research misconduct
No, it isn't misconduct - it's just that the situation could be nicer.
It would be academic misconduct if the code at the time of writing and submitting their publication on the authors' machines had not produced the results described (incl. crashing and not producing any results).
- writing code that is not portable or
- fragile in the sense that it is quite likely to break when its surroundings (OS, interpreter, dependencies) evolve and
are not scientific misconduct even if it is not "best programming practice".
Trust in their results?
That's a difficult one.
- On the one hand, with the possible exception of computer science, the scientific ability of the authors may be quite uncorrelated to their software development abilities (see linked answers above)
- On the other hand, if their software doesn't calculate what they think (claim) it does, then also the scientific content may be compromised.
For the case in question:
I tend to think that crashes (or stopping with an error message) are often relatively harmless in terms of scientific integrity of the paper. Short of blatant misconduct (claiming results that weren't obtained) this points to the code not being robust/maintained in an evolving environment (or data/formatting subteties) and not necessarily an incorrect implementation on their system.
I'd be more concerned about the lack of unit tests confirming the results of calculations that actually run through: statistics offers lots of possibilities to have logical error which lead to wrong but often even plausible numbers. That's what is scary to me...
What to do
Be extremely nice to them.
In many fields, it is not (yet) standard to publish code at all. They did you a favor in giving you an implementation that presumably works on their machine. In fact, as long as it takes you less time to fix & test their package than to write & test your own implementation of their paper, you have a net gain!
I don't want to insinuate that you are not nice to them.
However, my experience of some 10 years as (more or less active according to the ressources described above) maintainer of an R package is that the majority of help requests is answered with RTFM and many bug reports do not provide a minimal working example that I can reproduce and there are a few insistent and obnoxious help requests that suck up your time like a black hole because they try to offload the time they should put into learning R onto you package maintainer. While I try hard to treat all requests in a professional, friendly and timely manner, I also sometimes fail in that (most often fail is: timely). (And I have to say that those bad experiences are offset by also receiving extremely well written issue reports, sometimes with immediately usable pull requests and finding online contributors whom I'd otherwise never have known and even occasional "thank you for this package" emails. But there's a whole lot of truth to bad interactions producing far more impression than good interactions.)
You may just have had the bad luck to happen to be the one after a couple of toxic requests and/or may have inadvertendly triggered such an alarm with the authors.
Again while that is not the ideal interaction, you best bet to get the issue resolved is to be so supernice as to convince the authors that not all interactions on the internet are bad... Gives you good karma, too.
Encourage collaboration and make it easy for them: do put in the time to dig into those lines for debugging. Or write an R workaround and ask them to be so kind as to review your suggested changes.
In order to get the code up and running, ask them whether they can provide you a
sessionInfo () of their system and the versions of the relevant underlying libraries. The logical step after you trying several configurations that didn't work is to try and get a reproduction of "their" system to work.
Of course, it is completely fine if you decide to stop sinking time into a not-so-well designed package and either
- write your own implementation of the model described in their paper, or even
- completely abandon that model if you distrust the science behind it.