66

You should write down your thoughts now, then get on and do the proper research, before trying to do any experiment, if you wish to be taken seriously. While there is a need for radically different ideas, most research succeeds by following what has been done before ('standing on the shoulders of giants'). It is very unlikely that you will come up with ...


63

It would be dishonest to do this without mentioning that the algorithm was tuned differently. You ought to specify what the tuning changed and how this affects the results of the algorithm. You should also list accuracy results for the fast algorithm and speed results for the accurate algorithm. (Your probably also want some numbers for middle of the road ...


52

To paraphrase your question, "Is it honest to suggest that my algorithm is both fast and accurate when, in fact, it can only be fast and not-so-accurate or accurate and not-so-fast?" NO!!! Of course it isn't. Seriously, why do you even need to ask?


32

In many fields, finding good methodology was a long and tedious process. You gain nothing by repeating this process, if you can manage in one life time at all. However, if you do not use a reasonable methodology, your research will likely be without value. For example, consider that you have twenty drugs and try to investigate whether any of is good for the ...


16

I am just a Master student, so I do not know much of the dynamics of “the game”. Therefore I can only give some spectator opinion. One of my supervisors likes to have brutally honest plots in his papers. His work focuses on the scaling of parallel algorithms. For starters, he chooses strong scaling instead of weak scaling. The former is taking a fixed ...


16

Examples are areas where a study takes advantage of a sudden, unexpected event. For example, there have been studies about human memory related to unexpected tragedies. There was an idea that lots of Americans remembered exactly where they were when they found out that President Kennedy was shot. After the Challenger disaster, a study was done where they ...


11

First, some (hopefully) obvious remarks: you only need to present CPU specs if you are going to publish CPU timings CPU timings depend on variety of factors, including hardware specs, OS, drivers, libraries, software, configuration of this software, etc. It is unlikely that the combination of all these factors will be reproduced exactly. Even a very ...


10

The amount of experimental evidence that is needed in order to adequately demonstrate a result depends strongly on the current state of the art that you are comparing against. In some cases, even a single instance will be interesting and publishable, as there is much knowledge that can be gleaned from it. See, for example, the discovery of homo naledi: ...


10

This is impossible to answer without reading the actual paper, but I will do my best to give you an indication. If the hypothesis is exciting and the simulation is well-executed and also gives the expected results: probably publishable. Otherwise, probably not. Given that you need to ask this question here, it looks more like a "no", but don't let me (or ...


8

When you run an experiment that is expected to have some variability (because it is not in a perfectly controlled environment) then you must run it a large number of times. Not "until you get the results that are consistent with what you expect" but "until you get results that are reasonably consistent with one another." Then, when you describe the results, ...


8

No, you shouldn't do the experiment until you get the desired result. This is poor science and borderline unethical. (I'm sure you had no ill intent and that you are trying to deal with real-world complications.) An experiment is flawed if it is run n times but only the results from n - 1 are used and reported. (Or anything less than n). See Peter Norvig'...


8

You will almost certainly have to recruit study participants yourself. As others have suggested, speak with your advisor to out whether there are any participant pools he's used in the past. If you find others that he hasn't used before it's a good idea to make sure he's OK with the pool before you dive in, so to speak. Note that, in some fields, the pool ...


7

Criterion validity concerns with measuring the right thing. For instance, GPA is likely to have criterion validity to measure a student's academic understanding. While the change in weight in the last semester has much less criterion validity to measure the same trait. Basically, if the measurement you use and the trait you want to measure has a high ...


7

I don't see any ethical problem in not compensating people for work they don't do. However, you still have to frame the problem in a clear way in order to avoid frustration/litigations. Just write down explicitely that a student will receive compensation only if he/she completes the test. As for motivating students to give you meaningful answer, that is a ...


7

Can I use default parameters from the existing methods? Yes, but should you? Default parameters are unlikely to provide optimal performance, so you may like to try various parameters.


6

It somewhat depends of the publication style that you use for your thesis. For example, APA (6th edition) does not recommend to cite standard office software and programming languages. Therefore, if you would follow the APA guidelines, you'd just mention Python everywhere you need without citing any sources, however, you'd have to cite the MATLAB software. ...


6

You're probably thinking of PlanetLab. PlanetLab is a global research network that supports the development of new network services. Since the beginning of 2003, more than 1,000 researchers at top academic institutions and industrial research labs have used PlanetLab to develop new technologies for distributed storage, network mapping, peer-to-peer ...


6

Compare apples with apples Algorithm performance is rarely evaluated in isolation: usually, different algorithms are compared to one another or to some reference algorithm. When doing such a comparison, you should determine conditions in which reference algorithms were evaluated, and evaluate your own algorithm in the same conditions: if reference ...


6

This is highly dependent on what field you're in. I'm guessing physics from the question, but even so it depends on your subfield. Still, as a general matter of scientific methodology: there have been plenty of papers - including many good ones, and many famous ones - that have raised hypotheses and presented models, without also including experiments. This ...


5

I am not aware of any university run participant pools. I am aware of many departmental run and research group run participant pools. Some charities and companies also run participant pools. It is quite possible that you will not have to recruit the participants yourself. Your supervisor and/or colleagues would be a good place to start asking about ...


5

I am currently aware of two large research networks of this type: EmuLab allows experimentation with large numbers of virtual machines co-located within a facility at the University of Utah. GENI is a large-scale distributed networking testbed that also allows reconfiguration of the network switching hardware. Both are primarily sponsored by the US ...


5

Actually, I had a programming part in my masters thesis in which I wanted to show that some parameters do not have significant impact on the results. This is what I did which seems to be a combination of your methods: I drew a plot with all the parameter remaining the same except one parameter that I want to study on. Even if the variation of the parameter ...


5

I did my PhD in experimental nuclear structure physics, in a similar situation to the one you've described here. Within a month of starting my PhD, I was sent to the IFIN-HH lab in Romania for a week or so, to help a collaborator with their experiment. A few months later, I traveled to the experiment which provided my PhD data at Argonne National Laboratory ...


4

You need to ask the international students' office at your university, because It's a delicate legal issue of the type they are trained to navigate, and The answer may depend on the particulars of how the university handles its students and how your experiment handles compensation, and The answers to these questions change frequently, as visa and ...


4

Quoting the results from another paper (with citation) is all right, but using it in the benchmark comparison table is something which is practiced in research? I would wager it is not done enough. It is certainly ok, even required, to compare your results to earlier work, and the more fair and unbiased you can do this, the better. It is perfectly ...


4

I don't think there's anything wrong with voicing your concerns, but if you're going to you need to find a way to do it constructively while sticking to the facts. "Much disappointment and headache" might be entirely correct, but this is more of an emotional comment. Also: are there other students you can get to join you in voicing this concern? Maybe ...


4

This question is only vaguely related to Academia, honestly; the tips we can give you here apply equally well to virtually any project you'll ever run. First thing: given that you're new at this, you will probably screw up something at some point, so just internalize that. You're new, the field is new, and things go wrong when its a first time. Most likely ...


4

I have served on the IRB at two research institutions. The first question I would ask is what does your consent form state about how subjects are compensated if they withdraw? As of this point, if you do not indicate such in your consent form, you should compensate the subjects what you promised (whether they withdraw or not) until the end of the study or ...


4

In my view this depends on a number of things: a) Are the default parameters of the competing methods advertised as being widely applicable? b) Have authors of competing methods provided appropriate guidance for how to tune? c) Does what you are proposing require a lot of tuning, and did you tune yours in order to beat the others? Answers a) yes b) no c) ...


Only top voted, non community-wiki answers of a minimum length are eligible