Skip to main content
added 13 characters in body
Source Link
Alexandros
  • 19k
  • 3
  • 50
  • 84

Both previous answers are excellent. First of all, you should be very happy that you discovered the error BEFORE publication. Errors do happen, but you should also acknowledge the fact that there probably was an error in your workflow. Speaking from a CS prespective, for any new problem introduced in a paper, one must clearly describe what are the expected results when solving this specific problem. AFTER clearly defining the problem and its expected results then you must try to find a method that tackles it.

For this reason, many CS papers propose a naive benchmark method for any new problem, which might be super slow but it guarantees 100% that it gives out correct results. Then the same papers propose an "optimized" version that does the same thing, but in a much better optimized way. Although in the paper the selling point will be the fast method (and how faster it is compared to the naive version), during the "debugging" phase the naive version is very important because it always gives you the correct results. That means that everytime you do an optimization, parallelization or something similar to your proposed method, you must always check that you get the exact same results with the naive method. That guarantees that no errors are introduced at any phase of your implementation which leads to a better paper, safer assumptions and correct results.

Speaking from a personal perspective, even after writing many paperspublishing more than a handful papers and having done the coding in almost allmost of them (and knowing by now what usually works and what not), any time I add a new major optimization that makes the "optimized" version, e.g. 10% better, I almost always get a subtle mistake in the results (that might give wrong results in 3 out of 1000 experiments). Again, these errors are easy to spot because after any major code change, I always compare results to the naive method. Then I fix the "subtle" bug, get the correct results in all cases and move-on to the next optimization. Rinse, repeat.

Regarding your case, for a bug to go unnoticed for 6 months, it is obvious that you probably had no "naive" method that gave you the correct results to compare your results with. This is a big mistake that is mostly done by inexperienced researchers. We all should have confidence in our abilities to think of a better method to tackle problems in our area of expertise but thinking a correct idea and actually implementing it to be correct are two different things. You should always have a naive method that 100% gives correct results, before starting working on your "optimized" method. This slows down experiments (because the naive method might require several hours to run for all your test cases) but it is your safety belt when you are speeding towards a better solution. That way, errors are easily spotted within a few days or weeks and are easy to fix, many weeks before even starting to work on the paper. Moreover, it shows early if the method you thought has any true potential, before losing another six months to make it better. Also, it makes you confident that what you write in your paper is correct and no unpleasant surprises will happen, when other people try to recreate your results. Thus, next time, when you will work on another problem and a new paper always write a slow, bullet-proof method that gives correct results (checked manually one-by-one) before writing the new optimized version. The will protect you from losing another six months or publishing a paper based on wrong assumptions.

Both previous answers are excellent. First of all, you should be very happy that you discovered the error BEFORE publication. Errors do happen, but you should also acknowledge the fact that there probably was an error in your workflow. Speaking from a CS prespective, for any new problem introduced in a paper, one must clearly describe what are the expected results when solving this specific problem. AFTER clearly defining the problem and its expected results then you must try to find a method that tackles it.

For this reason, many CS papers propose a naive benchmark method for any new problem, which might be super slow but it guarantees 100% that it gives out correct results. Then the same papers propose an "optimized" version that does the same thing, but in a much better optimized way. Although in the paper the selling point will be the fast method (and how faster it is compared to the naive version), during the "debugging" phase the naive version is very important because it always gives you the correct results. That means that everytime you do an optimization, parallelization or something similar to your proposed method, you must always check that you get the exact same results with the naive method. That guarantees that no errors are introduced at any phase of your implementation which leads to a better paper, safer assumptions and correct results.

Speaking from a personal perspective, even after writing many papers and having done the coding in almost all of them (and knowing by now what usually works and what not), any time I add a new major optimization that makes the "optimized" version, e.g. 10% better, I almost always get a subtle mistake in the results (that might give wrong results in 3 out of 1000 experiments). Again, these errors are easy to spot because after any major code change, I always compare results to the naive method. Then I fix the "subtle" bug, get the correct results in all cases and move-on to the next optimization. Rinse, repeat.

Regarding your case, for a bug to go unnoticed for 6 months, it is obvious that you probably had no "naive" method that gave you the correct results to compare your results with. This is a big mistake that is mostly done by inexperienced researchers. We all should have confidence in our abilities to think of a better method to tackle problems in our area of expertise but thinking a correct idea and actually implementing it to be correct are two different things. You should always have a naive method that 100% gives correct results, before starting working on your "optimized" method. This slows down experiments (because the naive method might require several hours to run for all your test cases) but it is your safety belt when you are speeding towards a better solution. That way, errors are easily spotted within a few days or weeks and are easy to fix, many weeks before even starting to work on the paper. Moreover, it shows early if the method you thought has any true potential, before losing another six months to make it better. Also, it makes you confident that what you write in your paper is correct and no unpleasant surprises will happen, when other people try to recreate your results. Thus, next time, when you will work on another problem and a new paper always write a slow, bullet-proof method that gives correct results (checked manually one-by-one) before writing the new optimized version. The will protect you from losing another six months or publishing a paper based on wrong assumptions.

Both previous answers are excellent. First of all, you should be very happy that you discovered the error BEFORE publication. Errors do happen, but you should also acknowledge the fact that there probably was an error in your workflow. Speaking from a CS prespective, for any new problem introduced in a paper, one must clearly describe what are the expected results when solving this specific problem. AFTER clearly defining the problem and its expected results then you must try to find a method that tackles it.

For this reason, many CS papers propose a naive benchmark method for any new problem, which might be super slow but it guarantees 100% that it gives out correct results. Then the same papers propose an "optimized" version that does the same thing, but in a much better optimized way. Although in the paper the selling point will be the fast method (and how faster it is compared to the naive version), during the "debugging" phase the naive version is very important because it always gives you the correct results. That means that everytime you do an optimization, parallelization or something similar to your proposed method, you must always check that you get the exact same results with the naive method. That guarantees that no errors are introduced at any phase of your implementation which leads to a better paper, safer assumptions and correct results.

Speaking from a personal perspective, even after publishing more than a handful papers and having done the coding in most of them (and knowing by now what usually works and what not), any time I add a new major optimization that makes the "optimized" version, e.g. 10% better, I almost always get a subtle mistake in the results (that might give wrong results in 3 out of 1000 experiments). Again, these errors are easy to spot because after any major code change, I always compare results to the naive method. Then I fix the "subtle" bug, get the correct results in all cases and move-on to the next optimization. Rinse, repeat.

Regarding your case, for a bug to go unnoticed for 6 months, it is obvious that you probably had no "naive" method that gave you the correct results to compare your results with. This is a big mistake that is mostly done by inexperienced researchers. We all should have confidence in our abilities to think of a better method to tackle problems in our area of expertise but thinking a correct idea and actually implementing it to be correct are two different things. You should always have a naive method that 100% gives correct results, before starting working on your "optimized" method. This slows down experiments (because the naive method might require several hours to run for all your test cases) but it is your safety belt when you are speeding towards a better solution. That way, errors are easily spotted within a few days or weeks and are easy to fix, many weeks before even starting to work on the paper. Moreover, it shows early if the method you thought has any true potential, before losing another six months to make it better. Also, it makes you confident that what you write in your paper is correct and no unpleasant surprises will happen, when other people try to recreate your results. Thus, next time, when you will work on another problem and a new paper always write a slow, bullet-proof method that gives correct results (checked manually one-by-one) before writing the new optimized version. The will protect you from losing another six months or publishing a paper based on wrong assumptions.

added 310 characters in body
Source Link
Alexandros
  • 19k
  • 3
  • 50
  • 84

Both previous answers are excellent. First of all, you should be very happy that you discovered the error BEFORE publication. Errors do happen, but you should also acknowledge the fact that there probably was an error in your workflow. Speaking from a CS prespective, for any new problem introduced in a paper, one must clearly describe what are the expected results when solving this specific problem. AFTER clearly defining the problem and its expected results then you must try to find a method that tackles it.

For this reason, many CS papers propose a naive benchmark method for any new problem, which might be super slow but it guarantees 100% that it gives out correct results. Then the same papers propose an "optimized" version that does the same thing, but in a much better optimized way. Although in the paper the selling point will be the fast method (and how faster it is compared to the naive version), during the "debugging" phase the naive version is very important because it always gives you the correct results. That means that everytime you do an optimization, parallelization or something similar to your proposed method, you must always check that you get the exact same results with the naive method. That guarantees that no errors are introduced at any phase of your implementation which leads to a better paper, safer assumptions and correct results.

Speaking from a personal perspective, even after writing many papers and having done the coding in almost all of them (and knowing by now what usually works and what not), any time I add a new major optimization that makes the "optimized" version, e.g., 10% better, I almost always get a subtle mistake in the results (that e.g., givesmight give wrong results in 3 out of 1000 experiments). Again, these errors are easy to spot because after any major code change, I always compare results to the naive method. Then I fix the "subtle" bug, get the correct results in all cases and move-on to the next optimization. Rinse, repeat.

Regarding your case, for a bug to go unnoticed for 6 months, it is obvious that you probably had no "naive" method that gave you the correct results to compare your results with. This is a big mistake that is mostly done by inexperienced researchers. We all should have confidence in our abilities to think of a better method to tackle problems in our area of expertise but thinking a correct idea and actually implementing it to be correct are two different thingsthinking a correct idea and actually implementing it to be correct are two different things. You should always have a naive method that 100% gives correct results, before starting working on your "optimized" method. This slows down experiments (because the naive method might require several hours to run for all your test cases) but it is your safety belt when you are speeding towards a better solution. That way, errors are easily spotted within a few days or weeks and are easy to fix, many weeks before even starting to work on the paper. Moreover, it shows early if the method you thought has any true potential, before losing another six months to make it better. Also, it makes you confident that what you write in your paper is correct and no unpleasant surprises will happen, when other people try to recreate your results. Thus, next time, when you will work on another problem and a new paper always write a slow, bullet-proof method that gives correct results (checked manually one-by-one) before writing the new optimized version. The will protect you from losing another six months or publishing a paper based on wrong assumptions.

Both previous answers are excellent. First of all, you should be very happy that you discovered the error BEFORE publication. Errors do happen, but you should also acknowledge the fact that there probably was an error in your workflow. Speaking from a CS prespective, for any new problem introduced in a paper, one must clearly describe what are the expected results when solving this specific problem. AFTER clearly defining the problem and its expected results then you must try to find a method that tackles it.

For this reason, many CS papers propose a naive benchmark method for any new problem, which might be super slow but it guarantees 100% that it gives out correct results. Then the same papers propose an "optimized" version that does the same thing, but in a much better optimized way. Although in the paper the selling point will be the fast method (and how faster it is compared to the naive version), during the "debugging" phase the naive version is very important because it always gives you the correct results. That means that everytime you do an optimization, parallelization or something similar to your proposed method, you must always check that you get the exact same results with the naive method. That guarantees that no errors are introduced at any phase of your implementation which leads to a better paper, safer assumptions and correct results.

Speaking from a personal perspective, even after writing many papers and having done the coding in almost all of them (and knowing by now what usually works and what not), any time I add a new optimization that makes the "optimized" version e.g., 10% better, I always get a subtle mistake in the results (that e.g., gives wrong results in 3 out of 1000 experiments). Again, these errors are easy to spot because after any major code change, I always compare results to the naive method. Then I fix the "subtle" bug, get the correct results in all cases and move-on to the next optimization. Rinse, repeat.

Regarding your case, for a bug to go unnoticed for 6 months, it is obvious that you probably had no "naive" method that gave you the correct results to compare your results with. This is a big mistake that is mostly done by inexperienced researchers. We all should have confidence in our abilities to think of a better method to tackle problems in our area of expertise but thinking a correct idea and actually implementing it to be correct are two different things. You should always have a naive method that 100% gives correct results, before starting working on your "optimized" method. This slows down experiments (because the naive method might require several hours to run for all your test cases) but it is your safety belt when you are speeding towards a better solution. That way, errors are easily spotted within a few days or weeks and are easy to fix, many weeks before even starting to work on the paper. Moreover, it shows early if the method you thought has any true potential, before losing another six months to make it better. Also, it makes you confident that what you write in your paper is correct and no unpleasant surprises will happen, when other people try to recreate your results.

Both previous answers are excellent. First of all, you should be very happy that you discovered the error BEFORE publication. Errors do happen, but you should also acknowledge the fact that there probably was an error in your workflow. Speaking from a CS prespective, for any new problem introduced in a paper, one must clearly describe what are the expected results when solving this specific problem. AFTER clearly defining the problem and its expected results then you must try to find a method that tackles it.

For this reason, many CS papers propose a naive benchmark method for any new problem, which might be super slow but it guarantees 100% that it gives out correct results. Then the same papers propose an "optimized" version that does the same thing, but in a much better optimized way. Although in the paper the selling point will be the fast method (and how faster it is compared to the naive version), during the "debugging" phase the naive version is very important because it always gives you the correct results. That means that everytime you do an optimization, parallelization or something similar to your proposed method, you must always check that you get the exact same results with the naive method. That guarantees that no errors are introduced at any phase of your implementation which leads to a better paper, safer assumptions and correct results.

Speaking from a personal perspective, even after writing many papers and having done the coding in almost all of them (and knowing by now what usually works and what not), any time I add a new major optimization that makes the "optimized" version, e.g. 10% better, I almost always get a subtle mistake in the results (that might give wrong results in 3 out of 1000 experiments). Again, these errors are easy to spot because after any major code change, I always compare results to the naive method. Then I fix the "subtle" bug, get the correct results in all cases and move-on to the next optimization. Rinse, repeat.

Regarding your case, for a bug to go unnoticed for 6 months, it is obvious that you probably had no "naive" method that gave you the correct results to compare your results with. This is a big mistake that is mostly done by inexperienced researchers. We all should have confidence in our abilities to think of a better method to tackle problems in our area of expertise but thinking a correct idea and actually implementing it to be correct are two different things. You should always have a naive method that 100% gives correct results, before starting working on your "optimized" method. This slows down experiments (because the naive method might require several hours to run for all your test cases) but it is your safety belt when you are speeding towards a better solution. That way, errors are easily spotted within a few days or weeks and are easy to fix, many weeks before even starting to work on the paper. Moreover, it shows early if the method you thought has any true potential, before losing another six months to make it better. Also, it makes you confident that what you write in your paper is correct and no unpleasant surprises will happen, when other people try to recreate your results. Thus, next time, when you will work on another problem and a new paper always write a slow, bullet-proof method that gives correct results (checked manually one-by-one) before writing the new optimized version. The will protect you from losing another six months or publishing a paper based on wrong assumptions.

Source Link
Alexandros
  • 19k
  • 3
  • 50
  • 84

Both previous answers are excellent. First of all, you should be very happy that you discovered the error BEFORE publication. Errors do happen, but you should also acknowledge the fact that there probably was an error in your workflow. Speaking from a CS prespective, for any new problem introduced in a paper, one must clearly describe what are the expected results when solving this specific problem. AFTER clearly defining the problem and its expected results then you must try to find a method that tackles it.

For this reason, many CS papers propose a naive benchmark method for any new problem, which might be super slow but it guarantees 100% that it gives out correct results. Then the same papers propose an "optimized" version that does the same thing, but in a much better optimized way. Although in the paper the selling point will be the fast method (and how faster it is compared to the naive version), during the "debugging" phase the naive version is very important because it always gives you the correct results. That means that everytime you do an optimization, parallelization or something similar to your proposed method, you must always check that you get the exact same results with the naive method. That guarantees that no errors are introduced at any phase of your implementation which leads to a better paper, safer assumptions and correct results.

Speaking from a personal perspective, even after writing many papers and having done the coding in almost all of them (and knowing by now what usually works and what not), any time I add a new optimization that makes the "optimized" version e.g., 10% better, I always get a subtle mistake in the results (that e.g., gives wrong results in 3 out of 1000 experiments). Again, these errors are easy to spot because after any major code change, I always compare results to the naive method. Then I fix the "subtle" bug, get the correct results in all cases and move-on to the next optimization. Rinse, repeat.

Regarding your case, for a bug to go unnoticed for 6 months, it is obvious that you probably had no "naive" method that gave you the correct results to compare your results with. This is a big mistake that is mostly done by inexperienced researchers. We all should have confidence in our abilities to think of a better method to tackle problems in our area of expertise but thinking a correct idea and actually implementing it to be correct are two different things. You should always have a naive method that 100% gives correct results, before starting working on your "optimized" method. This slows down experiments (because the naive method might require several hours to run for all your test cases) but it is your safety belt when you are speeding towards a better solution. That way, errors are easily spotted within a few days or weeks and are easy to fix, many weeks before even starting to work on the paper. Moreover, it shows early if the method you thought has any true potential, before losing another six months to make it better. Also, it makes you confident that what you write in your paper is correct and no unpleasant surprises will happen, when other people try to recreate your results.