What are the publication options for academic contributions that don't lend themselves to a journal article?

Question

What are the publication options for academic contributions that don't lend themselves to the journal article format? In the case of datasets, there are some suggestions here, but I'm also wondering about other types of resources, such as

• software packages for scientific computing, and
• presentations of information that include interactive elements that are integral to the usefulness of the resource.

It seems to me that a common practice is for creators of such resources to publish an accompanying article, such that there is an avenue through which the resource can be 'cited'. But my impression is that often such articles do not add much value beyond that of the original resource. So a two part question:

1. Are there other options for 'publishing' such resources directly?
2. Are there pathways for having such resources peer-reviewed?

With regard to interactive resources, I'm thinking of a broad range, from this kind of interactive exposition to, say, a formal rubric-like taxonomy with more than two dimensions, such that some interactivity is required to toggle between which pair of dimensions are displayed (assuming only a 2D representation is used). To have such resources peer-reviewed, the closest option I know of is Distill, an web-only journal for machine-learning that facilitates interactive elements in the articles it publishes. But, for now at least, that option is limited to contributions in machine learning.

Answer 1

What are the publication options for...software packages for scientific computing,

The software can be made available on the Internet, e.g., via GitHub. (A personal website could be used too, but GitHub has many advantages.) A tutorial for the software could be published using traditional channels.

and presentations of information that include interactive elements that are integral to the usefulness of the resource.

To include interactive elements you'll need a medium that can support such elements. Publishing a video on the Internet might suffice, e.g., on YouTube. (A personal website could also be used.)

---

My answer doesn't address the (new) peer-review aspect, which was added to the question after my answer was written.

Answer 2

It seems to me that a common practice is for creators of such resources to publish an accompanying article, such that there is an avenue through which the resource can be 'cited'. But my impression is that often such articles do not add much value beyond that of the original resource. Are there other options for 'publishing' such resources directly, or pathways for having them peer-reviewed?

I don't think you give articles enough credit. They have several upsides:

• They hook into the "citation score" mechanics that help establish your academic credentials. That might sounds shallow and self-serving. But if you worked hard and made something good, there's nothing wrong in wanting get credit for it.

• If someone uses your dataset for research, or uses your software package as a key tool, then they need to provide references. This allows readers of that research to see what it's based on and how it could be reproduced. So you are helping a lot if you make it easy for people using your work to provide a good reference to it. Now, you yourself say you're not sure how to publish it; the same goes for anyone wondering how to cite a dataset or package. But by tying it to an article, you make it a lot easier. Academics might not know how to cite a dataset or package, but citing an article is familiar to them. And finding an article is also more familiar than chasing down a dataset or package. So the article makes your work a lot more accessible.

• You can hand anyone a PDF or print of an article and they can read it through, and decide if it's relevant for them. That's something you can do in the hallway at a conference. Telling people to download a dataset or install a package and play around with it? Takes longer, and you need to be behind your own desktop. So an article is a more portable preview of your work.

• How good the article is, is really up to what you put into it as a writer.

So what should be in the article?

As I've already outlined, the article is sort of a preview of your work, helping people decide if they should download your work. It's also a map to help find your work.

For a dataset you could talk about:

• Where to find it. What's the stable URL?
• Explanation of what the data is, where it came from and how it was gathered. If there are ethical, confidentiality or privacy issues that someone using the data needs to know, also describe those.
• Statistical analysis of the data, like what the distribution is of different items in the data.
• What kind of outlier removal or data cleaning has been done.
• What kind of data types are used, are any transformations like binning done? What is the range of different values?

For a piece of software you could talk about:

• Where to get it.
• How your software is licenced.
• What kind of long-term maintenance users can expect.
• High-level indication of hardware requirements and software dependencies. People reading your article should be able to identify whether this is something that will run on their desktop PC, smartphone, or that it requires a supercomputer, and what kind of OS and programming languages will work.
• A high-level overview of the structure of the package.
• Where to find the detailed documentation.
• Discussion of any (scientifically) interesting design choices. For example, if you're implementing an algorithm but not all of the literature defines the algorithm in exactly the same way, make it clear which one you used.
• Discussion, code fragments or pseudocode explaining the key/original algorithms.
• If under the hood you use a different algorithm than what people would expect, you may want to include a theoretical proof showing they're equivalent.