Questions tagged [experiment-design]
Technical aspects, ethics, and accepted norms or standards related to designing, conducting, or reporting research. Also, questions on teaching research design.
67
questions
0
votes
0
answers
97
views
Retaining study participants in a longitudinal research study
Which methods can be followed to retain participants in a longitudinal research study?
We tried to retain participants by sending reminder messages, and also a bonus at the second time point, but none ...
28
votes
6
answers
6k
views
Are experiments described in scientific papers *actually* peer-validated today?
Background: the answers and comments on this question prompted this question.
I know that when a new scientific theory is published there is a rush for people to devise experiment to prove or disprove ...
-1
votes
2
answers
105
views
Alternative experiemental or evaluation methods for programming language construct (artefact) [closed]
I am writing my B.Sc. in Computer Science. I am currently writing my methods chapter and adhering to the IMRaD structure (https://writingcenter.gmu.edu/writing-resources/imrad/writing-an-imrad-report)...
- 101
4
votes
2
answers
302
views
Equivalent to agile software development but for research?
In software development, there are many development methodologies with a focus on speed/efficiency: Agile, Scrum, DevOps, Kanban etc.
Are there similar methodologies in scientific research? (something ...
- 499
1
vote
1
answer
303
views
Concerned PhD student used employee for experiments and didn't acknowledge their work
Okay, So I'm not sure if I am making a big deal out of nothing but I am concerned based off of what I have heard so I figured I'd ask for advice.
I recently left a R&D job at a small private ...
2
votes
1
answer
288
views
How do you benchmark against another paper's result if their code is not released, and my replication of their method obtain different results? [duplicate]
I've developed an algorithm that I believe should theoretically improves upon a previous work. In the previous work, the authors have provided results for a baseline method and their own method. The ...
- 23
8
votes
3
answers
1k
views
Including tests done at a commercial lab in a manuscript
In the course of an experimental study, I sent a few samples for testing at a commercial lab. By commercial, I mean that they provide standardized testing services for a fee, and are not engaged in ...
- 13.2k
0
votes
1
answer
48
views
Defining the type of simultaneously retrospective and prospective cohort study [closed]
I'm planning a study design that starts collecting patients from 2015 to 2021, but as a three-year follow-up is necessary, their outcome measures will be recorded up to 2024. Thus, how should I define ...
- 103
6
votes
2
answers
370
views
My experiment supports a theory. Shall I contact the theorist attaching with my drafts?
We did an experiment and the results unexpectedly support a famous theory. I am not entirely sure how to write the full paper or sell the results because I am not too familiar with that theory. And I ...
- 4,292
0
votes
1
answer
77
views
What questions can be put into a survey to prevent low-quality responses? [closed]
I'm designing a survey and I heard that there's a technique that sets up some questions in the middle of the survey to see if people are actually reading these questions.
Could anyone provide me the ...
- 29
1
vote
1
answer
152
views
Is this an experimental or observational study? [closed]
We are conducting a study to validate an educational game on PC. A number of subjects are invited to use the game to go inside a virtual office room and put markers on the places where they "like&...
- 99
2
votes
1
answer
137
views
How much is the relevance of a synthetic benchmark?
Often researchers verify their proposed algorithm or techniques using standard benchmarks. My question is related to performance of a benchmark application.
If a researcher propose a new performance ...
- 153
2
votes
1
answer
92
views
When doing comparative evaluation among different research tools should we debug others research tool?
Researchers propose software tools as part of their research work to gather some metrics about application performance. I intend to propose a tool with better features and capabilities. For comparison ...
- 153
3
votes
1
answer
125
views
Why is it considered sub-par to re-test something that has been shown before as opposed to addressing a knowledge gap?
Firstly, apologies if this is the incorrect StackExchange child to post this one. The only other study design questions I could find were here.
So...
Common feedback I get as I plan different ...
- 330
4
votes
2
answers
143
views
Are user surveys considered "human experiments" by the U.S. National Science Foundation?
I'm applying for a grant to the National Science Foundation of the United States of America.
The grant is to fund our research in numerical methods for fashion. I want to conduct a survey where I ...
- 491
3
votes
1
answer
159
views
How can I pre-register my study of little importance?
I am writing an undergraduate dissertation on behavioural economics. I am looking to run an experiment. As nobody apart from my course convenor will read my paper, the stakes are not particularly high....
- 131
3
votes
2
answers
569
views
Is a software-simulated experiment enough? [closed]
My ideal experiment to test my hypothesis would involve launching a large apparatus into deep space - and would cost millions of dollars, which I dont have.
Instead, if I develop a software model/...
- 251
5
votes
6
answers
1k
views
Is a hypothesis necessary for empirical research?
Suppose I would like to do an experiment that compares two existing methods for solving a certain problem, and decides which method is better (in some predefined metric).
I can start with a ...
- 16.6k
9
votes
1
answer
770
views
What are examples of experiments or studies where pre-registration would not be feasible?
The incoming editor of Psychological Science mentions in an interview that pre-registration isn't feasible for all submissions to the journal:
In instances in which preregistration is feasible... ...
- 1,323
1
vote
1
answer
184
views
How to learn to make a meaningful study without having an academical background?
I am an autist who had been in therapie for 8 years. In that therapie I learned a lot about my own and later about others psychology. I also developed over the years an tendency to just experiment ...
- 111
0
votes
1
answer
116
views
How to include experiments vs state-of-the-art papers with no code
I am working a paper which offers an algorithmic improvement, which is measured in both the quality of the results, and the time it took to achieve them.
There are 3 recent (relatively) papers which ...
- 111
3
votes
3
answers
704
views
Fine-tuning parameters for existing methods [closed]
I'm comparing my method against existing methods by doing some experiments on one dataset.
The existing methods do not provide results on the considered dataset.
My question is: Can I use default ...
- 295
-1
votes
1
answer
40
views
Is it valid to aggregate multiple participant recruitment sources? [closed]
For example, let's say you use Amazon Mechanical Turk for participant recruitment. What if you now also use online message boards? Can you aggregate the data together as long as the procedure for both ...
- 1
0
votes
1
answer
207
views
Feasibility of using blockchain to avoid researcher bias [closed]
Would it be feasible, presently or in the future, to use computers with blockchain technology to help avoid researcher bias?
For instance, in a medical experiment, when an experimenter uses ...
- 19
5
votes
1
answer
167
views
Do scientists using user facilities (such as Advanced Light Sources) have to travel frequently?
I recently met an experimental physicist who does his measurements using the X-ray beam-line at a national lab. Since the university he works at is not too close to the facility, I wonder how much ...
- 153
3
votes
2
answers
326
views
Author contributions include statements about experiment not conducted in study
I am a graduate student submitting a computational biology paper for peer review. It is intended to be a “methodology paper” for the journal. Our paper uses new methodology to analyze several public ...
- 133
3
votes
0
answers
72
views
Is it usually allowed to use lie detecting technique (maybe using neural activities) during post-experimental inquiry?
When an experimenter want to find out about e.g. what is the hypothesis perceived by the subjects via post-experimental inquiry, if the subjects answered that they did not perceive any hypothesis, it ...
- 1,002
2
votes
3
answers
300
views
Can Research Hypotheses be Contratictory Among Them?
I want to know how to proceed in an experimental research, where there is a conflict between different theories with different predictions. I.e. the theory A predicts X > Y, while the theory B ...
- 21
2
votes
0
answers
129
views
Is apprenticeship/reading the only way to learn experimental design? Are there books/papers about the theory of designing a sound experiment?
I am slogging away at grad school and I have a nagging feeling I can be more efficient. I feel like a person who cannot see more than a foot in front of them so I hit a lot of walls. My planning and ...
- 762
0
votes
5
answers
146
views
Sampling question for telephone survey. [closed]
Health sciences researcher out of my area of expertise. I have an assistant calling businesses at random, using a publicly available registry, to ask about their automated external defibrillator (AED)....
- 73
0
votes
1
answer
51
views
Is it necessary to add experiment to my designed framework for a journal
I'm working a framework X and trying to publish it in IEEE Communication Magazine. But, due to its limitation on the number of text and figures, I wonder it is necessary to add any experiment or proof-...
- 298
7
votes
1
answer
2k
views
What computer specs should be included in numerical results section, when, and why?
Generally in the numerical section for papers I read, they include (if at all) computer specs like how much RAM, CPU speed, or type of processor. What kinds of computers specs should I include? When ...
- 173
37
votes
6
answers
8k
views
Is it reasonable to not study previous research to avoid bias?
I am currently a student at a tech school and otherwise unemployed. I’ve never been a scientist or published a paper.
So I am a little skeptical of some results I have been hearing about experiments ...
- 479
2
votes
0
answers
181
views
Is it possible to request a scientific experiment from a laboratory?
I have a scientific experiment that I need done to prove a theory of mine. Unfortunately I don't have the resources to perform said experiment like a fully functional laboratory such as Los Alamos ...
- 121
2
votes
3
answers
1k
views
Is using a similar experimental procedure and analysis structure to test something different considered plagiarism?
Here is the situation:
I am currently writing a thesis which uses a behavioral experiment to answer the main research question. Its main aim is to test how a certain welfare scheme would affect ...
- 21
-1
votes
2
answers
76
views
Experimental results summarized in a single large table OR smaller tables with subset of results?
In my experiment, I have obtained a large chunk of results for many different problems being effectively solved by my implemented hypothesis.
If I keep my results as subsets for each problem, then I ...
- 12.6k
0
votes
1
answer
428
views
Thesis experiment done by others
I started my masters with semi finished experiment by my supervisor, and h suggested to do more experiments to include them in my thesis.
However, after year and a half, the other methods we were ...
5
votes
2
answers
369
views
Reproducible and exact calculations vs. approximate and slightly random results in experiment
When experiments are performed and reported in an academic article, I learned that reproducibility is important. I am implementing a system where I stand between the choices to use GPU acceleration to ...
- 231
5
votes
3
answers
2k
views
Is is ethical to not compensate experiment participants who withdraw from the study?
I do experiments on learning and instruction for my dissertation. They are almost no risks in participating in our studies--except maybe boredom since they tend to run for 30 minutes to 1 hour. We get ...
- 1,539
5
votes
3
answers
186
views
Changing an experiment due to preliminary data
I'm writing a paper comparing the performance of 2 algorithms (lets call them A and B). A performs better than B, however preliminary data suggested a way to significantly improve the performance of B....
- 53
1
vote
1
answer
163
views
When a STEM research topic can be studied experimentally, computationally, and theoretically, do PhD programs typically focus on only one approach?
In American universities, when a STEM research topic can be studied experimentally, computationally, and theoretically, do PhD programs in this research topic only focus on one of the three approaches ...
- 1
0
votes
1
answer
172
views
Tips for Conducting an Experiment without Stress
I have been working independently on a research project for the past few months and I am almost ready to actually conduct the experiment. The experiment itself should only take two hours. However, I ...
- 603
5
votes
3
answers
1k
views
Disorganized and overworked professor causing disappoinment and headache
About two years ago I finished my master thesis. At our university we can only choose the topic to work on and mostly it is part of a Phd program.
My topic was a mess because the faulty study design ...
- 181
1
vote
1
answer
217
views
First time being in an applied math lab - what are some tips to overcome shyness?
This will be my first semester in an applied math lab. After spending a couple of years taking all theoretical math courses, essentially working in isolation the entire time, and finishing a thesis, ...
- 43
7
votes
1
answer
332
views
How to deal with an unhygienic human subject?
The human subject that I really depend on is extremely unhygienic. Not simply sweaty way, but full-on stench, black teeth and dirty clothes.
I feel responsible for the lab equipment and space that I ...
- 73
29
votes
6
answers
6k
views
How to tune algorithm performances in computer science papers
In computer science field is common to produce papers that present algorithms that estimate something with a certain accuracy and a certain speed.
Many algorithms can clearly be tuned to have high ...
- 3,250
1
vote
1
answer
137
views
design student assignments
In a recent interview, I have been asked how to design assignments for an undergraduate course on computer science. Based on my previous TA experience, I did help my supervisor to design students ...
- 208
0
votes
1
answer
163
views
"Fake" Things that are used in Experiments [closed]
What do you call the "things" that you used in experiment, but not in real life? Things that are made for experiment/simulation purpose. For example, phantom for human body.
- 111
4
votes
1
answer
398
views
Research paper with experimental results from a small amount of data
I have written a research paper in image processing that I will be submitting to an IEEE conference. I have proposed a new method to detect 'X' in images. However, due to hardware failure I could ...
- 693
9
votes
1
answer
588
views
Good first-time study pre-registration website/service
For my PhD thesis in artificial intelligence, I am designing a human study where subjects interact with multiple versions of an algorithm and rate their experience. I like the idea of pre-registering ...
- 2,809