Our paper has recently been accepted to a be presented at a workshop. It's a compiler paper that discusses how we generate code for a specific algorithm for a specific hardware accelerator. Unfortunately, that hardware accelerator won't be released to public soon, and our results are based on a simulator.

Two reviewers weakly accepted it, while the other two weakly rejected it. The main criticism is that while we say that we believe the ideas in the paper can be applied to public hardware, it's better to show experimental results on public hardware, which we didn't.

The honest answer to the question above is that I ran out of time to implement those ideas, and it wasn't in the company's best interest to spend time implementing that algorithm on different hardware.

The company found the code generator to be useful (at least as far as saving developer time goes, which was hard to quantify without a user study), but I'm not sure I can convince the public that these ideas are useful unless I reproduce the results on public hardware.

I was a research intern working for that hardware company when we wrote the paper. I don't work with the company any more. My PhD thesis focuses on a different topic, and I won't have the time to further carry out these experiments.

I have limited experience presenting papers. If somebody at the workshop comments or asks some version of the following questions, how should I answer them?

  • Why don't you have results on public hardware?
  • Do you have any plans to implement this on public hardware?
  • I'm not convinced that these results are useful unless they are carried out on public hardware.

If you can also anticipate similar questions, I'd also like to know how to address them.

My full-time colleagues may not be present at the workshop, and I also think that they won't have the time to implement this "future work" either.

I've looked for similar questions, but wasn't able to find one. Hope this is not a duplicate.

  • Just to give you the "true, but useless answer": You should answer the truth. – User Sep 26 '19 at 16:34

I think a good approach is to just say what you do here.

"We don't have more results because of time and (maybe) money."

"Yes, we would like to extend the results to public hardware (assuming you do)."

"I'm not convinced either until we can get the time and money to carry out the extended experiment."

Your results are valid only so far as your actual experiment can be validly extrapolated. There is nothing wrong with that. Treat the questions as an opportunity, not a complaint.

However, if there are aspects of commonly available hardware that might make it harder or easier to extrapolate, you could "surmise" a bit about that.

| improve this answer | |
  • 1
    "We don't have more results because of time and (maybe) money." Sounds unnecessarily defensive. Why not "We haven't had the chance to perform these experiments yet, but we agree that it's among the most important directions for future work." – lighthouse keeper Sep 26 '19 at 17:07

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.