I am writing a paper for my first conference which is actually about my MSc dissertation. In my dissertation I designed a system and tested it with a number of users however, I don't want to include the testing/results in the conference paper because I believe it needs more testing (on a larger number of users).

I'm not sure though if I can submit the paper without results or will it be rejected because of that?

  • 6
    Normally, you would include some preliminary results and convert your gut-feeling into a founded explanation as to WHY the system need more testing.
    – Alexander
    Apr 29, 2014 at 11:30
  • Often in the case of Computer Science (software engineering in particular), short papers in conferences are for what it seems you are describing: preliminary results.
    – user7112
    Apr 29, 2014 at 12:15
  • 2
    You certainly may submit the paper. It very well may be rejected. Apr 29, 2014 at 13:54
  • If the papers published in this conference in the past all include results, that's a signal that unexceptional submissions without results probably won't be accepted.
    – ff524
    Apr 29, 2014 at 14:08
  • I'd include your results as preliminary and explain why more data is required to obtain a significant result. Apr 29, 2014 at 14:53

1 Answer 1


(as you are talking about a system that needs to be evaluated on a number of users, I have the impression that you may be in a field related to software engineering, so that's what I am working with in the rest of the answer - but other fields will likely not be much different, just the examples I provide below may be of less relevance)

The answer to this question largely depends on what conference you plan to submit to, and how preliminary your results are. Different conferences value different things, and differently ranked conferences need results on different levels of maturity.

Some examples from the wider field of systems and software engineering:

  • ICSE is the major conference in software engineering. You will need a solid idea and a strong, convincing evaluation. Testing on a significant number of real users will be required (what this means exactly depends a bit on the type of system, of course). Generally, reviewers at ICSE are not so interested in the concrete software you built, more in the processes, concepts and methodologies underlying your system.
  • USENIX Middleware is another major conference, but with a completely separate focus. Here, the actual system implementation is what counts. Few Middleware papers have user studies etc., but all of them provide detailed performance analysis and comparisons with existing tools and systems. Providing your system in a useable (!) way for download is almost mandatory.
  • The World Wide Web Conference is the premier conference for Web engineering. What really counts here is the data you have. Access to real-life data (e.g., usage data, search logs, social network data sets, etc.) is really important, more so than testing with real users. Papers that provide a new publicly available set of real-life data with some interesting characteristics are often valued highly. Implementation generally counts less than algorithms.
  • ICSOC is an example of a smaller, more subfield-oriented conference. Here, you need a good idea and some preliminary or intermediary results, but most papers are not evaluated very strictly. Most papers submitted here are still, at least to some extend, work-in-progress.

Basically all sub-fields have their own smaller conferences similar to ICSOC (last example). While they are usually at least one tier below the more general conferences, they are usually a good first outlet for publication, as they are often less competitive and do not require a fully evaluated and finished research project. However, do not think that smaller conferences accept everything. ICSOC still sports an acceptance rate between 15% and 20%.

What I am trying to say is that different conferences work completely differently, and submitting the same paper to two (even equally good) conferences can easily end in a Strong Reject followed by Strong Accept. It is important to know how papers in a given conference typically look like, how they are presented, and how they are evaluated. Going against the mainstream in a conference is usually not the best idea, especially not in the beginning. This is were you need the advise of a senior person, who should have experience in writing papers for the relevant conferences in your field. If that is not available, you can also try to carefully survey previous proceedings, and see if you can find patterns and overall styles yourself. However, be prepared that you may miss nuances if you do this yourself. For instance, if your target conference typically does not have many applied papers, it is difficult to tell without knowing an experienced insider whether this is because applied papers are not often submitted there, or because the TPC always rejects almost all applied papers.


I noticed that I rambled on without ever really answering your questions ...

I'm not sure though if I can submit the paper without results or will it be rejected because of that?

Without any results may be difficult, although you can still try to submit to a workshop or work-in-progress track. However, if you already have some preliminary results, it sounds like a smaller, more specialized conference may be a good choice for you. Of course the validity of this depends a bit on how small your evaluation was...

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .