Skip to main content

Questions tagged [experiment-design]

Technical aspects, ethics, and accepted norms or standards related to designing, conducting, or reporting research. Also, questions on teaching research design.

Filter by
Sorted by
Tagged with
37 votes
6 answers
8k views

Is it reasonable to not study previous research to avoid bias?

I am currently a student at a tech school and otherwise unemployed. I’ve never been a scientist or published a paper. So I am a little skeptical of some results I have been hearing about experiments ...
Caston's user avatar
  • 479
29 votes
6 answers
6k views

How to tune algorithm performances in computer science papers

In computer science field is common to produce papers that present algorithms that estimate something with a certain accuracy and a certain speed. Many algorithms can clearly be tuned to have high ...
Ulderique Demoitre's user avatar
28 votes
6 answers
6k views

Are experiments described in scientific papers *actually* peer-validated today?

Background: the answers and comments on this question prompted this question. I know that when a new scientific theory is published there is a rush for people to devise experiment to prove or disprove ...
LorenzoDonati4Ukraine-OnStrike's user avatar
11 votes
3 answers
1k views

How to choose participants for a study?

I want to know how can I get some participants for an empirical research study I would be conducting. Does universities have systems in place to call/recruit participants or do I have to find them ...
Matir's user avatar
  • 833
11 votes
2 answers
197 views

Ref. request : book or survey about conducting experimental studies

I am currently conducting researches that are almost mainly theoretical (I am a TCS researcher gone rogue = I also work on applying theory stuff to real life problems). In the future I plan to conduct ...
Sylvain Peyronnet's user avatar
10 votes
1 answer
778 views

What are examples of experiments or studies where pre-registration would not be feasible?

The incoming editor of Psychological Science mentions in an interview that pre-registration isn't feasible for all submissions to the journal: In instances in which preregistration is feasible... ...
cjs's user avatar
  • 1,333
9 votes
1 answer
602 views

Good first-time study pre-registration website/service

For my PhD thesis in artificial intelligence, I am designing a human study where subjects interact with multiple versions of an algorithm and rate their experience. I like the idea of pre-registering ...
Martin Modrák's user avatar
8 votes
3 answers
1k views

Including tests done at a commercial lab in a manuscript

In the course of an experimental study, I sent a few samples for testing at a commercial lab. By commercial, I mean that they provide standardized testing services for a fee, and are not engaged in ...
AppliedAcademic's user avatar
7 votes
1 answer
2k views

What computer specs should be included in numerical results section, when, and why?

Generally in the numerical section for papers I read, they include (if at all) computer specs like how much RAM, CPU speed, or type of processor. What kinds of computers specs should I include? When ...
MathIsKey's user avatar
  • 173
7 votes
1 answer
333 views

How to deal with an unhygienic human subject?

The human subject that I really depend on is extremely unhygienic. Not simply sweaty way, but full-on stench, black teeth and dirty clothes. I feel responsible for the lab equipment and space that I ...
Scio's user avatar
  • 73
7 votes
3 answers
2k views

Difference between criterion validity and reliability equivalence

In the book Doing research in the real world by David E. Gray, there is a section on experiment design. When discussing validity and reliability, the author defines “criterion validity” as This is ...
rumtscho's user avatar
  • 4,121
6 votes
2 answers
370 views

My experiment supports a theory. Shall I contact the theorist attaching with my drafts?

We did an experiment and the results unexpectedly support a famous theory. I am not entirely sure how to write the full paper or sell the results because I am not too familiar with that theory. And I ...
High GPA's user avatar
  • 4,506
6 votes
1 answer
174 views

Do scientists using user facilities (such as Advanced Light Sources) have to travel frequently?

I recently met an experimental physicist who does his measurements using the X-ray beam-line at a national lab. Since the university he works at is not too close to the facility, I wonder how much ...
Sidd's user avatar
  • 163
6 votes
2 answers
237 views

How can I do better research when the experimental design is lacking?

I would like to do first-class research. However, how do I actually accomplish that when my supervisors design not-so-great experiments and I don't have the resources to do more independent work? In ...
fff's user avatar
  • 61
5 votes
3 answers
2k views

Is is ethical to not compensate experiment participants who withdraw from the study?

I do experiments on learning and instruction for my dissertation. They are almost no risks in participating in our studies--except maybe boredom since they tend to run for 30 minutes to 1 hour. We get ...
iamnarra's user avatar
  • 1,529
5 votes
2 answers
165 views

Large distributed research network

I remember having heard of a large distributed computer network being operated by several universities. As far as I remember, it would be possible to get several (hundreds) of virtual machines to ...
Florian's user avatar
  • 161
5 votes
3 answers
1k views

Disorganized and overworked professor causing disappoinment and headache

About two years ago I finished my master thesis. At our university we can only choose the topic to work on and mostly it is part of a Phd program. My topic was a mess because the faulty study design ...
CuriousIndeed's user avatar
5 votes
6 answers
2k views

Is a hypothesis necessary for empirical research?

Suppose I would like to do an experiment that compares two existing methods for solving a certain problem, and decides which method is better (in some predefined metric). I can start with a ...
Erel Segal-Halevi's user avatar
5 votes
3 answers
194 views

Changing an experiment due to preliminary data

I'm writing a paper comparing the performance of 2 algorithms (lets call them A and B). A performs better than B, however preliminary data suggested a way to significantly improve the performance of B....
user3362964's user avatar
5 votes
2 answers
508 views

Reproducible and exact calculations vs. approximate and slightly random results in experiment

When experiments are performed and reported in an academic article, I learned that reproducibility is important. I am implementing a system where I stand between the choices to use GPU acceleration to ...
Stephen Johnson's user avatar
5 votes
1 answer
651 views

What is the strategy to identify natural experiment as a PhD student?

As social scientists become more concerned with causal inference, a good thesis these days needs some sort of quasi-experimental strategy. For example, an economist wants to know the effect of ...
Heisenberg's user avatar
  • 1,476
4 votes
1 answer
402 views

Research paper with experimental results from a small amount of data

I have written a research paper in image processing that I will be submitting to an IEEE conference. I have proposed a new method to detect 'X' in images. However, due to hardware failure I could ...
kunal18's user avatar
  • 693
4 votes
2 answers
156 views

Are user surveys considered "human experiments" by the U.S. National Science Foundation?

I'm applying for a grant to the National Science Foundation of the United States of America. The grant is to fund our research in numerical methods for fashion. I want to conduct a survey where I ...
El Marce's user avatar
  • 543
4 votes
2 answers
345 views

Equivalent to agile software development but for research?

In software development, there are many development methodologies with a focus on speed/efficiency: Agile, Scrum, DevOps, Kanban etc. Are there similar methodologies in scientific research? (something ...
Alex I's user avatar
  • 519
4 votes
2 answers
674 views

Do researchers reuse benchmark results for existing algorithm when comparing with their algorithm?

Unfortunately I do not have any prior research experience, therefore I had a few questions while I was studying papers on cluster analysis. Can a researcher legally and with permission use the ...
phoxis's user avatar
  • 518
4 votes
2 answers
218 views

How to develop experiment to justify construction of a prototype system?

I have worked hard for my master thesis project to build a system that uses an already existing algorithm, just in a mobile setting, which hasn't been done before. Because the algorithm is complex, it ...
user2212461's user avatar
  • 1,631
4 votes
1 answer
173 views

How can I pre-register my study of little importance?

I am writing an undergraduate dissertation on behavioural economics. I am looking to run an experiment. As nobody apart from my course convenor will read my paper, the stakes are not particularly high....
Lieu Zheng Hong's user avatar
3 votes
2 answers
570 views

Is a software-simulated experiment enough? [closed]

My ideal experiment to test my hypothesis would involve launching a large apparatus into deep space - and would cost millions of dollars, which I dont have. Instead, if I develop a software model/...
Dingredient's user avatar
3 votes
3 answers
705 views

Fine-tuning parameters for existing methods [closed]

I'm comparing my method against existing methods by doing some experiments on one dataset. The existing methods do not provide results on the considered dataset. My question is: Can I use default ...
lenhhoxung's user avatar
3 votes
2 answers
4k views

How to mention an implementation tool in thesis?

In my MSc thesis implementation section, I have used two tools: Matlab and Python scripting. How should I introduce and mention them in my implementation chapter? Thank you,
svahidhoss's user avatar
3 votes
1 answer
89 views

Impact of parameters in experimental results of paper

I am writing a paper in computer science literature. I have a method that has two main parameters. It turns out that one of these parameters does not have a significant impact on the quality of the ...
orezvani's user avatar
  • 1,227
3 votes
1 answer
125 views

Why is it considered sub-par to re-test something that has been shown before as opposed to addressing a knowledge gap?

Firstly, apologies if this is the incorrect StackExchange child to post this one. The only other study design questions I could find were here. So... Common feedback I get as I plan different ...
sleepy's user avatar
  • 360
3 votes
2 answers
346 views

Author contributions include statements about experiment not conducted in study

I am a graduate student submitting a computational biology paper for peer review. It is intended to be a “methodology paper” for the journal. Our paper uses new methodology to analyze several public ...
user12211991's user avatar
3 votes
2 answers
422 views

Are there standard example research papers for teaching methodology and experiments?

Are there recommended or standard published papers (not books) that show great research methodology/ scientific methods? I am trying to help out a friend who is in business and they are trying to ...
user-2147482637's user avatar
3 votes
0 answers
72 views

Is it usually allowed to use lie detecting technique (maybe using neural activities) during post-experimental inquiry?

When an experimenter want to find out about e.g. what is the hypothesis perceived by the subjects via post-experimental inquiry, if the subjects answered that they did not perceive any hypothesis, it ...
Aqqqq's user avatar
  • 1,002
3 votes
0 answers
225 views

What tradeoffs are there between internal and external validity? [closed]

I was reading through an article about research design. The authors gave an example on the relation between internal and external validity which I thought might be an important area of consideration ...
Derrick Mah's user avatar
2 votes
3 answers
418 views

Can Research Hypotheses be Contratictory Among Them?

I want to know how to proceed in an experimental research, where there is a conflict between different theories with different predictions. I.e. the theory A predicts X > Y, while the theory B ...
Ágatha's user avatar
  • 21
2 votes
3 answers
1k views

Is using a similar experimental procedure and analysis structure to test something different considered plagiarism?

Here is the situation: I am currently writing a thesis which uses a behavioral experiment to answer the main research question. Its main aim is to test how a certain welfare scheme would affect ...
ELC's user avatar
  • 21
2 votes
1 answer
344 views

How do you benchmark against another paper's result if their code is not released, and my replication of their method obtain different results? [duplicate]

I've developed an algorithm that I believe should theoretically improves upon a previous work. In the previous work, the authors have provided results for a baseline method and their own method. The ...
Tianxun Zhou's user avatar
2 votes
1 answer
141 views

How much is the relevance of a synthetic benchmark?

Often researchers verify their proposed algorithm or techniques using standard benchmarks. My question is related to performance of a benchmark application. If a researcher propose a new performance ...
ajit's user avatar
  • 153
2 votes
1 answer
95 views

When doing comparative evaluation among different research tools should we debug others research tool?

Researchers propose software tools as part of their research work to gather some metrics about application performance. I intend to propose a tool with better features and capabilities. For comparison ...
ajit's user avatar
  • 153
2 votes
0 answers
130 views

Is apprenticeship/reading the only way to learn experimental design? Are there books/papers about the theory of designing a sound experiment?

I am slogging away at grad school and I have a nagging feeling I can be more efficient. I feel like a person who cannot see more than a foot in front of them so I hit a lot of walls. My planning and ...
TheCodeNovice's user avatar
2 votes
0 answers
192 views

Is it possible to request a scientific experiment from a laboratory?

I have a scientific experiment that I need done to prove a theory of mine. Unfortunately I don't have the resources to perform said experiment like a fully functional laboratory such as Los Alamos ...
Steve Mucci's user avatar
1 vote
3 answers
2k views

Should I repeat experiment affected by external factors until achieving consistent result

I am doing an experiment assumed to be performed in ideal environment although it is not, e.g. resource is fully allocated from a resource pool shared between different organisations. As my experiment ...
user12635's user avatar
  • 3,367
1 vote
1 answer
185 views

How to learn to make a meaningful study without having an academical background?

I am an autist who had been in therapie for 8 years. In that therapie I learned a lot about my own and later about others psychology. I also developed over the years an tendency to just experiment ...
Zaibis's user avatar
  • 111
1 vote
1 answer
319 views

Concerned PhD student used employee for experiments and didn't acknowledge their work

Okay, So I'm not sure if I am making a big deal out of nothing but I am concerned based off of what I have heard so I figured I'd ask for advice. I recently left a R&D job at a small private ...
Concerned 3rd Party's user avatar
1 vote
1 answer
157 views

Is this an experimental or observational study? [closed]

We are conducting a study to validate an educational game on PC. A number of subjects are invited to use the game to go inside a virtual office room and put markers on the places where they "like&...
Mu09's user avatar
  • 99
1 vote
1 answer
166 views

When a STEM research topic can be studied experimentally, computationally, and theoretically, do PhD programs typically focus on only one approach?

In American universities, when a STEM research topic can be studied experimentally, computationally, and theoretically, do PhD programs in this research topic only focus on one of the three approaches ...
User001's user avatar
1 vote
1 answer
139 views

design student assignments

In a recent interview, I have been asked how to design assignments for an undergraduate course on computer science. Based on my previous TA experience, I did help my supervisor to design students ...
John Smith's user avatar
1 vote
1 answer
341 views

How to recruit participants for online experiments?

We are currently doing a study where we thought it might be smart to try out the possibilities of recruiting participants from the Internet. I was thinking it might be best to ask more experienced ...
puslet88's user avatar
  • 569