5

I'm currently working on a minor problem in information extraction. The idea behind the system is relatively simple and contains small modifications of existing algorithms. However in the process, I ended up creating a whole new dataset of a different kind - which did not exist before. This dataset would foster new novel research in my subdomain (call it problem X).

How can I stress the quality of my data and upcoming innovations it can drive and inspire rather than worrying about the simplicity of my algorithms (and reiteration and experimentation of other non-novel ideas)? There has been lot of significant research in problem X recently and my method doesn't stand anywhere in comparison with state of the art.

However I expect some great upcoming works (from the whole research community) in Problem X in upcoming couple of years. If I don't publish this dataset, someone else would definitely produce something similar within a year (think of it as need of the hour) along with some hi-fi "mathy" solutions to Problem X though I just experimented with existing data mining tools and libraries.

I have one more week to go. What should I do that would improve my chances of getting this simple work on Problem X but with an awesome new dataset published?

EDIT:

  1. Raw data of this kind is available publicly in massive amount but considered as noise, though it's easy for human perception and conveys a lot of sense.

  2. My contribution in creating dataset is annotations and analysis (basic statistical distributions etc)

  3. My approach to solve Problem X is not mathematically complex - but at par with regular Tier-II papers being published in this area.

  4. There IS a need of this kind of systems in the industry. (any citations would disclose my solution)

  • "This dataset would foster new novel research in my subdomain". How do you know that? Data (in CS) is not created out of thin air. If you used well known public data to produce something extraordinary then the data-creation process is worth publishing. If you have access to data that nobody else has, then there is nothing to publish. What is your case? – Alexandros Jul 31 '15 at 6:47
  • Thanks @Alexandros . This comment gave me tips on how to present myself... – Aditya Jul 31 '15 at 7:19
  • Meanwhile, (1) The data is accessible to public, in seriously huge volume (2) Data of this kind is usually regarded as noise, though humans understand it very well. (3) Required manual annotations – Aditya Jul 31 '15 at 7:21
10

Make your paper a data paper, and add in the bit about the simple algorithm into the introduction. That way, you put your data front and centre.

And then see the answers to Data publication basics - where, why, how, and when should I publish my unpublished data? for how to release the data for the whole community to access: doing this is almost always a requirement for getting a data-paper published.

7

If I understand your approach correctly, you want to write a (weak) research paper basically as an excuse to publish a data set. Your hope is that people will review the paper and get persuaded by the valuable data to accept it despite relatively uninspiring technical content. Further, you hope that your data set will actually lead to substantial impact, even if the paper ideas themselves won't.

I have to say, I tried similar things in the past. It never worked. There are a number of issues:

  • People don't typically think like that. A reviewer tasked with reviewing your paper will foremost review the paper. (S)he may or may not even notice that your paper, if accepted, would also release an important data set to the wild. Even if (s)he does, it is unlikely that this will change the review outcome much.
  • Even if your paper passes through peer review, there is a high chance that your data set won't be noticed by the community if it is "hidden" in a weak paper. Nowaydays, most fields have way too many papers published to read, or even be aware of, all of them. Weak papers tend to be published, glanced over, and forgotten. Most researchers won't look for the data set published along a paper that they consider not very useful.
  • You say that "expect some great upcoming works" from the research community related to your data set. There is a good chance that, even if true, this is not very obvious to other researchers yet. Basically, for many researchers, it will not be enough to just give them a data set that has data for A, B, C, and (now new!) X. You also need to inspire them with a great new algorithm, technique, analysis, whatever that makes use of X in combination of A, B, and C that this new data actually opens up new avenues for research. If everybody already knows that X will be the next hot thing, and X is also not extremely hard or expensive to get, then I am wondering why you happen to be sitting on this elusive data set and other research groups are not.

What you really should do is think about this last item yourself, and then work on a great paper using your data. If you think you can't do parts of the paper project yourself, try to get collaborators on board. If you can't convince collaborators that this is valuable data / research, then ... well, then you need to revisit whether you are not overestimating the value of your data in the first place.

Alternatively, there are some venues that publish artefact-only papers, where the "paper" is basically just a description of the data set and the real value comes from an indexed and archived listing of data. One example in my field is the PESOS workshop, which used to run a "Quest for Case Studies". However, in practice, the impact of data sets published like this was generally extremely low, in part because of the problem I mentioned in my third item above. Publishing cool new data is often not enough - you also want to convince people that new, interesting research can be done with the data, and this works best by starting to do such research yourself.

  • "However, in practice, the impact of data sets published like this was generally extremely low" - possibly, this is community-specific, but my impression is that dataset papers in the Semantic Web community are quite well-received. Possibly, that is because all datasets published there generally share one of a few common data formats (or are already set up as ready-to-use public endpoints). – O. R. Mapper Jul 31 '15 at 7:21
  • @xLeitix That's what I explained. Other research groups have also just started working in this area - though I've been working on this for six months now - they are larger groups, have greater knowledge and expertise. My system is not as complex as they might get - but I have the benefit of starting early. Think of it as a race to novelty. – Aditya Jul 31 '15 at 7:47
  • @O.R.Mapper can you please explain a little more on "ready-to-use public endpoints"? I think you got what I'm trying to explain. – Aditya Jul 31 '15 at 7:48
  • @Aditya: In the Semantic Web field, SPARQL is the de-facto standard query language. With such a uniform standard in place, arbitrary datasets can be made accessible on public URLs that anyone can send SPARQL queries to and retrieve results from, which in turn means that datasets published like that can directly be used via standardized interfaces without any import or conversion. – O. R. Mapper Jul 31 '15 at 7:55

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.