5

I am a freelancer software developer and I developed only commercial applications so far. I am thinking of publishing a paper and I would like some help.

Is re-rewriting the core function of a world class - well known open source project (where performance is a crucial) and optimizing it to be 2-3 times faster, considered research?

I am talking about using the exact same computer and exact same input and compare the performance. The idea is to parallelize it as much as possible, modify (a bit) well known data structures to be faster for the specific case, use a more suited language and (a bit) more suited algorithms than the open source project used.

In my view, I am applying what others made. I don't create an entirely new algorithm or data structure. But on the other hand, I guess there are 1,000s of papers published out there and many of them add little value, while my work will help improve the open source project.

What do you say, is the above considered "research worth publishing"?

Improve this question
4
  • Short answer since I've seen plenty of HPC papers and gotten one paper published about parallelisation: most likely not. If you want to publish on the mere basis of performance improvements, in most domains you will need several orders of magnitude of improvement. If your new algorithm design has other structural qualities and you can explain why they are beneficial, or if it is obvious how your method can further be improved to reach the wanted orders of magnitude of improvement, then you should be able to publish it.
    – Steve Dodier-Lazaro
    Sep 30, 2015 at 1:51
  • Exaggerating slightly: Computer science often considers improvements uninteresting unless they exceed an order of magnitude or change the entire complexity curve. (Hence all the gags about pessimizing compilers...) Software engineering might be more interested, if the solution is general and scales well.
    – keshlam
    Sep 30, 2015 at 3:18
  • "in most domains you will need several orders of magnitude of improvement" - well ... the more user-oriented people in my subfield frequently mock the more hardware-oriented people in the same subfield exactly because to the latter, changing a digit several places behind a comma in execution times is a publishable achievement. I suppose the actual improvement required depends strongly on the community.
    – O. R. Mapper
    Sep 30, 2015 at 7:19
  • If the open source project you mention is applied to a field outside of computer science, then it might be more easily published in a journal relating to that field (probably only if you are going to make the software itself publicly available though). For example, in bioinformatics it's not unusual to publish a tool which is basically a faster version of an already available tool (mostly for programs which take hours or days to run). However, if your main incentive is to improve the open source project then this might be achieved more easily by offering your services to that project instead.
    – ping
    Sep 30, 2015 at 21:10

1 Answer 1

Reset to default
8

Is re-rewriting the core function of a world class - well known open source project (where performance is a crucial) and optimizing it to be 2-3 times faster, considered research?

It might be, but only under certain restrictions:

  • You need to be sure no-one has written about the improvement before. And I am not talking about the particular piece of code in the particular project you are looking at here, but about the conceptual idea of how to improve the code. If you want to present your work as a scientific contribution, it needs to be conceptually novel in a way that no-one has described the methods you use in an abstract way before.
  • You need to present your improvement on an abstract, generalized level. Modifying a particular project is nice, but it is not research. Showing a novel method how to modify any number of projects that feature a particular problem can be.
  • Your change should not be entirely obvious. While research contributions do not necessarily need to be totally unexpected or groundbreakingly obscure ideas, if many developers could make the same modifications to the project simply by applying their professional knowledge, your contribution is probably not considered valuable enough, even if no-one else has described the actual procedure before.

To answer concretely to your points:

The idea is to parallelize it as much as possible,

This might be interesting for a paper if it involves any methods for parallelization that were previously unknown, and that can be applied to other applications, as well.

modify (a bit) well known data structures to be faster for the specific case,

A "specific case" is usually not interesting for research results; it only becomes interesting if you can show the "specific case" is generalizeable in some way.

use a more suited language

If you design a new language that is specifically suited for the type of applications in question, that could be publishable as research, at least if that language has conceptual advantages over existing domain-specific or general purpose languages for any class of applications. The mere translation into an existing language is most probably not publishable as research.

and (a bit) more suited algorithms than the open source project used.

If those algorithms are already known, it is probably somewhat obvious that they could be used on that particular proejct. As such, this is not a research contribution.

I am applying what others made.

Yes, fine.

I don't create an entirely new algorithm or data structure.

Well, do you create anything conceptually new at all?

But on the other hand, I guess there are 1,000s of papers published out there and many of them add little value, while my work will help improve the open source project.

"little value" is relative. In research publications, value does not mean a solution can readily be used. Rather than that, value is usually seen as the product of

  • the degree by which a problem is solved/situation is improved if the presented knowledge is applied
  • the amount of cases in which that problem occurred/to which the solution is applicable

Due to the second item, whether or not you already apply your method to one particular project is hardly relevant. As soon as it is known how to solve the issue, the research is done. Implementing the solution in a project may be helpful to the users of the project, but it does not advance the knowledge of humanity and as such, does not warrant a research publication in and of itself.

To summarize: A research publication is not about what has been done, but about what has been learned.

Improve this answer
5
  • 2
    Very interesting, thank you very much! Do you know where I can read more on the matter and decide if what I am doing "has been done before"? The big question in my mind, is "how new" my discovery needs to be?
    – SuperHeroY
    Sep 29, 2015 at 19:15
  • Search google scholar for case study in software optimisation or similar. I would find this interesting however you make the information public. What techniques did you use to diagnose performance? e.g. Profiling? And what patterns did you use as part of your solution? Anything unusual or unique?
    – Brendan Cody-Kenny
    Sep 29, 2015 at 20:18
  • 1
    I'm unsure in this particular area, but some conferences accept "application papers", with somewhat relaxed conditions about the contents... might be useful.
    – Fábio Dias
    Sep 29, 2015 at 22:23
  • 1
    @SuperHeroY: That is tricky - your hunch that "there are 1,000s of papers published" is right, but appending "every year, in each subfield" makes the dimensions of this more obvious. Getting an idea of the existing body of knowledge is a challenging task and one of the most important reasons why doctoral candidates are at first guided by a professor and postdocs who can point them into the right directions. This is even slightly trickier as each of your modifications might belong to a different subfield. Unless you can establish contact to someone who is already doing research in one of ...
    – O. R. Mapper
    Sep 29, 2015 at 22:29
  • ... the touched upon fields, trying to find out the main publication venues (which journals, which conferences) for the respective topics and then finding papers, e.g. via Google Scholar, as suggested by @McOda, is indeed the way to go.
    – O. R. Mapper
    Sep 29, 2015 at 22:29

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .