15

In the past I used to use a lot of Python/R/MATLAB which are the standard in many scientific disciplines. However, recently I have dived into Julia and now everything I'm doing is in Julia (including workshops, blogs, etc. to help teach and promote it!). And there are a lot of very good reasons to use Julia for these kinds of projects.

However, if I am to be publishing algorithms which have Julia code, should I be expecting backlash from reviewers? Should I have alternative versions (like a MATLAB version) also included until Julia gains more acceptance? I am curious as to whether other's past experience with using new tools was a headache, or whether reviewers tend to find it as a little extra novelty.

The field is numerical analysis, more specifically developing new computational methods for stochastic (partial) differential equations. It's written in Julia because from my experience I get massive performance increases over other scripting languages while requiring minimal coding/debugging time. In the paper, everything is written as psudocode for legibility. Note that R/Python/C have language binding tools which can allow someone (with some work) to use the code as a blackbox, but MATLAB users would have to re-implement the code themselves (the langugages are quite similar)

  • 2
    Regardless of how the reviewers view it, if you want the code to be accessible to a wide range of people, it would be good to have an additional implementation in a more common language. – Bitwise Apr 23 '16 at 15:46
  • 2
    Can you state in which community you are trying to publish? In many fields of CS, reviewers of scientific papers hardly ever look at code that the paper refers to. So stating a single sentence on why the language has been chosen in the paper would be enough to convince reviewers that what you did makes sense. And for using your code as a black-box, one certainly does not need to know Julia. – DCTLib Apr 23 '16 at 16:01
  • 4
    It depends on the journal you are submitting to. In some very CS'y journals, old algorithms in new languages is nothing more than the academic version of code golf. For example, to take something known to work in C and make it work in Python. However, if you write a totally novel analysis in Python, or a new tool for doing X (where X was previously possible, but now easier/faster/better in Python/Julia), then that's a different story, and is worthy of publication. In short, no one cares what language it was written in, unless that makes a significant practical difference. – Wetlab Walter Apr 23 '16 at 16:03
  • 5
    It's not CS as much since it's not "tool making", as much as it is development and testing of algorithms — Excuse me? What? – JeffE Apr 23 '16 at 20:31
  • 2
    You know what I meant. It's not "that kind" of CS that's related to package development and efficient code, instead it's numerical analysis (and yes, there is a blurred line between them). Some papers in the field even omit any code / numerical experiments (which yes, algorithms and theory parts of CS do as well, but you get the point of what I am trying to distinguish between) – Chris Rackauckas Apr 23 '16 at 21:53
16

I tend to get annoyed reading a paper that has its code examples in an atypical language for the area. I don't want to have to learn the language to understand the examples. Unless the paper is about the language and its features or advantages or disadvantages, the example code should be as straightforward to understand as possible, and using a toy or hobby language to provide algorithm examples makes the review harder to do. I'm likely to decline to even do the review if I don't think I'll be able to get through it easily. For example, working in high performance computing (my area), unless the paper is about the advantages of a new run-time system with its own language, I don't want to see some sparse matrix algebra routines in Haskell or Julia since I don't come close to speaking or understanding either of those languages. I'd rather you use a made up psuedo-code description of the implementation and not even provide the actual code unless you're making a specific point about the implementation that's part of the new results.

  • 5
    As clarification, in the paper I write everything in psudocode. However, I plan to give all of my implementations (in Julia) as well. Also, do you really consider Julia as a "toy or hobby language"? I'm getting the feeling it's really growing in HPC/Machine Learning communities. But in general, I interpret your answer is a clear "yes, include MATLAB/R/Python code whenever possible" which is disheartening but good to know. – Chris Rackauckas Apr 23 '16 at 18:24
  • 8
    I don't think the world has concluded anything about Julia yet. I'm not yet convinced, and as best I can tell, we don't see much use of it yet on our HPC resources. It's a hobby language for now, maybe we'll see what happens when something written in it wins a Gordon Bell Prize. Frankly, if the pseudo-code was there, I probably wouldn't look at an appendix of any code as a reviewer unless it was important to the details of the paper, in which case, I think it should be in the paper itself. – Bill Barth Apr 23 '16 at 18:33
  • 4
    Also, I might use "hobby language" and "research language" uncharitably interchangeably. – Bill Barth Apr 23 '16 at 18:57
  • 1
    "Atypical for the area" is bad enough, aside from the other value judgments. One of the best papers I have ever read on nonlinear computational mechanics (back in the 1980s) had the numerical algorithms in Algol 68. I can't imagine what substances the authors were taking when they made that choice. Neither can I remember any other occasion when it was worth the effort of learning a new language for the sole reason of understanding a paper! – alephzero Apr 24 '16 at 4:31
2

I think that the answer to the question in the title is that the programming language plays no role.

Concerning the question if you should use Julia or/and other languages, I think that the answer depends on your goals. If you head to reproducibility, then just using Julia is totally OK. Julia is simple enough to install and fairly simple to read, so that most people who are really interested should have no problem to test your code. If, however, you want that your code is actually used by others, you should provide an implementation in the "native language" of the community. Providing an additional implementation in Julia to promote the language (and I agree, Julia has the potential to become important in many computational fields) is helpful.

  • Had to adapt the first sentence after the title of the question was edited... – Dirk Apr 23 '16 at 19:37
2

There's two questions here. One is about what language you should write in for the purposes of your paper (and the source code related to you paper). The other is how much you want people to adopt your code.

The first is likely to have a small amount of impact on whether your paper gets accepted - in general it's the algorithm that's important, and your implementation is an existence proof and something you can measure performance on. The other consideration is not to scare your audience: don't make them do more work that they have to.

The adoption story is about how many citations your paper gets. To pick a random example, Numerical Recipes has 115,000 citations on Google Scholar. If you write your code with good documentation under an appropriate licence in a language that's accessible and people want to use, they'll use it and you'll get more citations. If you write in an esoteric language that's less likely to happen. Of course, the trick is guessing what the language of the next 20 years might be.

  • Adding to this, why would you write something performance-critical in an interpreted language? And why would I want to adopt your code, when I need a method to solve PDEs quickly? Best case, I might think it's worth translating your logic to C (as I would with pseudocode), but that still requires me to invest my time. – jamesqf Apr 24 '16 at 4:51
  • 3
    @jamesqf I am not sure I understand your remark -- Julia is not interpreted, and its performance should be in the same ballpark as C. – Federico Poloni Apr 24 '16 at 10:16
  • @Federico Poloni: I'm not familiar with Julia, but the OP says it's a scripting (therefore interpreted) language. – jamesqf Apr 30 '16 at 5:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.