Is using synthetic datasets in sciences the same as using fake data? [closed]

Question

Sometimes, there are good reasons to use synthetic data in research. In my case I am using data from Synthea, a program that simulated patient data including medications, names and addresses. I know that no hospital would hand me such real data, so there is no alternative. In my thesis I want to reveal specific patterns in patient records.

My idea is to test some statistical methods and neural networks to predict which disease a patient will have when he or she comes to the doctor next visit. Therefore a known statistical process in generating those patterns makes sense as I want to test some methods. I also want to infer which factors are the most important ones to predict the next disease. Therefore a mass of data is needed.

I am still not sure if using such data is useful in research. On the one hand, data protection or confidentiality in my case I would rate as positive cases using fake data, same as simulated data. As I have an ordinary private laptop it is a good idea to work with artificial datasets, as there is no need for special data security and data protection. Medical data are very sensitive and should not be stored on students' computers. Artificial data is also good for testing code and programs, as one knows what is going on under the hoods if the data generating source is documented well.

On the other hand, using faked data is considered as betraying sciences a lot of papers are retracted or removed when using faked data. Data fabrication is also listed as a scientific misconduct incident on the English Wikipedia.

What should I do?

Answer 1

Synthetic data and fabricated data are different. If you want to use synthetic data, you need to be transparent about how the data is generated and not tailor the synthetic data to support your hypothesis. Fabrication would be false data that is designed to support your hypothesis.

Remember that any statistical analysis will be a reflection of the process used to generate the data. It might be that the trend in the synthetic data reflects real data, but you would have to make that case by examining how the synthetic data is generated.

Answer 2

There is a use-case for synthetic data, but writing a thesis when dataprotection prohibits you from using real is not such a case.

An example of a use-case would be when the organization that does have access to the real data can create synthetic data with patterns from the real data, and give you that synthetic data. This could be all the "data" they can give you, or as an example dataset on which you can prepare your analysis before going on-site at the protected computer where you have access to the real data.

If you just create synthetic data without any relation to real data, then what would you hope to get out of such an analysis? I assume you heard of GIGO.

Answer 3

If you are using synthesized data and not disclosing that in the finished work, then you are committing academic misconduct. This is essentially the same as using fabricated data.

If you are using synthesized data and disclosing that the data is synthesized, then there is nothing dishonest or unethical about what you are doing. It is not the same as using faked or fabricated data, and not academic misconduct. However, it may be bad science - just because something is not unethical doesn’t mean it is a good idea or a valid approach to doing research. Depending on what you are trying to do with the synthesized data and how the data was produced, your results may have some usefulness, or they can be completely worthless.

In any case, you should not decide to use synthesized data without consulting your advisor. And you should never try to publish anything using data that is presented as real when it is simulated, or that is presented as collected or obtained in some way when it wasn’t.

Answer 4

Sometimes there are good uses for synthetic data in research --- this does not sound like one of them

The project you have described in your question does not sound sensible to me. "Synthetic data" (also called "simulated data" in some fields) is useful primarily as a way of illustrating how statistical inference methods and other modelling methods perform. For this purpose it is not necessary to use a real dataset. However, if the goal of your project is to reveal specific patterns in the variables, then all you are doing is determining the patterns that were generated in the synthetic data. Since patterns in synthetic data can be constructed however you want, what is the point of that?

Presumably the people at Synthea can specify the exact data-generating mechanism, so the patterns that arise in the data can be described deterministically. So you need to ask yourself whether there is really a need for inference about these patterns. If you want to use simulated data I would recommend that you refocus on illustrating the methodological aspects of your work --- e.g., showing that you have developed a good inference method that could be applied to other data.

As to the possibility of "betraying science" with "faked data", there is a world of difference between simulated data (which would be disclosed in the analysis) versus faked data (which occurs when a person attempts to pass off made-up data as real data). Any project using simulated data should make it clear that these data are simulated, and ensure that the reader does not confuse the simulated data with real data.