Probably, precise communication guidelines will (and should) vary between different fields, depending in part on how well machine learning methods have been described, utilized and benchmarked in prior works within the field. However, in my opinion, every empirical field should already be able to take note of lessons learned within computer science, and also implement sanity checks on results based on both general and field-specific knowledge. Indeed, it's not all that different from applying statistical methods to a given study.
The main new challenge is guaranteeing reproducibility while using ML methods. See e.g. M. Hutson: Artificial intelligence faces reproducibility crisis, Science (2018). (Non-paywall link here.) The article stresses the need of providing code, test data, and details on training procedures, but does not provide very specific guidelines.
However, Joelle Pineau, professor of computer science at McGill, has been pushing for higher reproducibility standards in machine learning-related research. Although not published in the traditional sense, the Reproducibility Checklist on her website is the closest to a standard guideline I know of. For example, it was used for submissions to the 2019 NeurIPS conference. (A report on how this worked out can be found here.) It gets routinely updated, so I recommend checking the source. However, I will reproduce some of the points that are relevant for empirical science:
For all datasets used, check if you include:
- The relevant statistics, such as number of examples.
- The details of train / validation / test splits.
- An explanation of any data that were excluded, and all pre-processing step.
- A link to a downloadable version of the dataset or simulation environment.
- For new data collected, a complete description of the data collection process, such as
instructions to annotators and methods for quality control.
For all reported experimental results, check if you include:
- The range of hyper-parameters considered, method to select the best hyper-parameter
configuration, and specification of all hyper-parameters used to generate results.
- The exact number of training and evaluation runs.
- A clear definition of the specific measure or statistics used to report results
- A description of results with central tendency (e.g. mean) & variation (e.g. error bars).
- The average runtime for each result, or estimated energy cost.
- A description of the computing infrastructure used.
This was based on version 2.0 of the checklist, dated April 7, 2020. Note that experimental above does not mean experiment in the sense of empirical science, but in the sense of running a "numerical experiment" on some set of data.
Finally, note that this is a developing field. I fully expect more guidelines to be written over the next few years, as people realize the need for it. For a sign that this coming, see e.g. this 2019 DOE report, which declared scientific reproducibility in applications of scientific machine learning a priority research direction.
