12

Often I have found that research students who have learned to do experiments have not learned to use software to handle their data effectively. It's not within my power to insert the appropriate training into their prior education. Teaching people to program or use software is not within my area of expertise or a priority for me. I have observed other research supervisors expecting their students to "teach your self to use what I use" but I realize that what I use, while very powerful, may not be best for a beginner. Also, I picked my tools a long time ago so the state of the art may have changed. What should I recommend to my students, and why? I would like for students to rapidly acquire flexible, durable analysis skills.

closed as off-topic by Cape Code, Brian Tompsett - 汤莱恩, gman, David Richerby, Bill Barth May 10 '16 at 15:44

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • ""Shopping" questions, which seek recommendations or lists of individual universities, academic programs, publishers, journals, research topics or similar as an answer or seek an assessment or comparison of such, are off-topic here. (See this discussion for more information.)" – Cape Code, Brian Tompsett - 汤莱恩, gman, David Richerby, Bill Barth
If this question can be reworded to fit the rules in the help center, please edit the question.

  • 4
    What kind of analysis do they need to do? And what volume of data? – Davidmh May 10 '16 at 10:42
  • 7
    This might be better asked at Software recommendations. They should be able to deal with academia-related requirements just fine. Consider flagging your question and requesting migration. – Stephan Kolassa May 10 '16 at 11:18
  • @StephanKolassa First rule of migration is whether it's off-topic at the current site. I think it should stick to Academia. – Mast May 10 '16 at 15:44
15

I would like for students to rapidly acquire flexible, durable analysis skills.

Your criteria are quite stringent! I think you are going to have to compromise at some point along the line. If you want them to acquire the skills rapidly, then they are probably going to have to use menu-based software, which will be limited in its flexibility. The long-term and more flexible solution would be for the students to learn statistical programming, but that of course has a steep learning curve.

In my opinion, R has a lot of advantages. [I imagine you have already come across it, so I may be stating the obvious here and you may have a good reason for ruling it out, but...]

  • It is free and open source, and therefore once learnt, the skill can be taken anywhere.

  • It's massively flexible when you take into account all of the add-on packages

  • Students can "ease" into it using R commander, which gives a menu-based interface but also outputs the corresponding code.

  • It is popular and therefore very well resourced.

The best compromise that I can think of would be to start the students off using the menu-based R commander package, but encourage them to inspect and customise the code where possible. If you are not able to give training yourself, it would probably be a good idea to arrange for someone else (either in your department, or pay someone external) to give a course. There are lots of good self-learning resources available, but a course ought to speed up the learning process. When they see how powerful the software is, it is likely to encourage them to put in the time and effort to learn to use it well.

  • 9
    R is a good answer only if the OP asks about statistics. In general, you can do whatever R does and more with, for example python and pandas. – Ander Biguri May 10 '16 at 12:10
  • 7
    Also, keep in mind that R has a rather steep learning curve. Unless you are handling lots of statistics, Python is a better choice for a general programming language to perform data analysis (and also do stats). It's also free, open source, and its usage is growing fast across disciplines. – Gabriel May 10 '16 at 12:31
  • 3
    @AnderBiguri I disagree with the first part of your statement. R is perfectly capable of handling lots more than just statistics. – user2390246 May 10 '16 at 13:06
  • 5
    @user2390246 true but R is quite unwieldy as a general programming language. Unless you are doing statistics, it is a poor choice in my opinion (I say this as a fan of R). – user24098 May 10 '16 at 13:10
  • 1
    Having recently self-taught both R and statistical python, it felt like the free resouces for R were considerably more accessible than the resources for numpy/pandas. – Racheet May 10 '16 at 13:36
16

Teaching how to program should be a priority in research, in at least Sciences. Doesn't matter what STEM field you are in, almost certainly you will need to deal with data, and using "black box" software only teaches to do whatever the software tells you.

I've seen people give results using standard deviation and mean for non-normal distributions, and its just because they didn't know how to plot their distribution and just used black box software. I've seen people rename file by file a folder with 500 files with data. These are worse than "Attack ships on fire off the shoulder of Orion".

Since nowadays almost all research in STEM is performed using computers, understanding computers is a must.

My recommendation is to make people learn MATLAB if you have access to it, or Python if you don't have access to MATLAB or you are a Open source / free software supporter. Both of this languages are designed to be very high level, and not need "advanced" computer science skills (such as inheritance in OOP, or pointers in C)*. Both of the languages are widely used and there are numerous free online courses to learn, in Coursera, Codeacademy, Udacity, EDx or any other online learning platform.

Learning how to code to the basic point should take less than 2 months, considering that meanwhile the student is also doing other things. And they can save thousands of hours of tedious work.

Let me repeat the key message: We need researchers with programming skills. Its incredibly important skill to be able to perform research in the XXI century.

While this answer mainly focuses in STEM, basic programming in other fields that use statistics is also useful.

*Of course, knowing about that helps.

  • 1
    I don't know about the absolute need for programming skills. Things like Excel are very powerful and do not need programming skills, some understanding of scripting lets you do a lot already. People who don't know how to stats will not learn by learning how to code. – Cape Code May 10 '16 at 13:00
  • 8
    @CapeCode, Excel has many limitations for data management and manipulation, and it is a poor choice for anything other than very simple data. While not everyone needs programming skills, they are increasingly important in a growing number of fields. The lack of training in this area is a big gap in the sciences. – user24098 May 10 '16 at 13:26
  • 4
    @CapeCode I have used excel for sciences because my advisors wanted it. It would take me about 10 times longer to make something than in Matlab, because excel is not a tool designed for science. No one says you can not use notepad to write papers, its just not the right tool for it. – Ander Biguri May 10 '16 at 14:08
  • 1
    While programming is definitely useful for handling large datasets (I do a bit of coding myself for my bioscience research), for small datasets (such as a single qPCR run, or a few cages of mice), specialised programs such as Excel or R are much more useful. – March Ho May 10 '16 at 14:47
  • 2
    @CapeCode the statistic deficiency was just a simple example, that would definitely be better addressed if the person would know what he/she was doing instead of using a black box software, which is the point. – Ander Biguri May 10 '16 at 14:48
8

We're talking about students here, not currently practicing researchers, and so my comment is really made with respect to future trends rather than the current state of play which I believe the other answers address.

I believe that in the future, more and more people will be expected to know how to program if they are going to do any kind of data analysis. Perhaps not on the more theoretical side, but since you said your students are doing practical work, I will assume that is not an issue. Tools like R and Matlab are good places to start if you are unfamiliar with programming and want to get something done right now; but honestly, since the barrier to entry for programming in fully-fledge generic programming languages is so low these days (and expected to get lower), I see no reason not to point students in the direction of a full programming language and the modules they might want for doing statistical analyses that are relevant to their field.

Whilst R and Matlab are fine choices, personally, I would introduce my students to something like Python, and the excellent modules that are avalible to do all the data analysis that can be done in R/Matlab that exist in the Python ecosystem. Python has a very gradual learning curve at the beginner end of the spectrum, while at the other end advanced programmers can write code thats just as fast as C if they take advantage of the newer, optimized interpreters. These 2 pros, plus the plethora of modules for doing any kind of analysis/plotting R or Matlab can do, is what has made Python the defacto language of choice in my field (Bioinformatics), and likely a powerful tool under your student's belts going forward with whatever they decide to pursue in life.

Of course, there are other languages out there, such as Java, Julia, Rust, etc - however I would rather teach those as second or third languages to learn, once you have a strong foundation in Python.

For the record, i'm not saying "teach them python", i'm saying just make them aware of it's existence.

3

Matlab/Octave is appropriate for physics (experimental)

Matlab is very well documented. and get all the graphs done you need as a physicist. It will allow your students to focus on the physics problem instad of hunting bugs or documentation.

Be sure that your university includes the statistics toolbox in the license.

1

Often I have found that research students who have learned to do experiments have not learned to use software to handle their data effectively

What kind of data handling and what kind of software program you're referring to? Is it programming language that analyze experimental data (R, Python/C++) or software (like SPSSS/Minitab, Matlab/Octave, Atlas.ti/VUE etc)?

R, C++ would be have a very steep learning curve. Even Python would not an easy entry for student who have no basic programming in their undergraduate years (based on my personal experience, it's not the syntax itself, but the early step on choosing and install IDE, configure module etc but maybe that just me).

This may sound cliche but for not starting flame wars about which software/program is the best, it all depends on the needs and background of the users. For example, those who has background in statistics would like to use R (R can do lot more than statistics, I know) but for biology students who do research in fermentation, maybe Minitab is enough. Same thing with Python/Ruby/Julia or C++/Fortran or Word/LaTeX etc.

Ask them what they learned in their undergraduate years and see if they can fully maximize it in their research. For me, the first step is not to learn fancy software but to have a clear understanding how to do research and a good research practice (research methodology, workflow, raw data management etc) and only then introduce software/program to help them.

1

The specific platform is going to be discipline specific so what will work for my students may not be best for your students.

However, if you teach them a proper digital-data-analysis workflow, then they will be able to transfer good habits to whatever specific analysis tool they are using.

For my students, I emphasize:

1) proper data archiving - each data set contains only data stored as plain text

2) proper metadata - each data set has accompanying detailed metadata

3) using code for data analysis - all data manipulations are completed with reproducible code and the original data are never altered

4) version control - we use git but that's just my preference

  • 1
    This is very good general advice, but not sure that it really answers the question. – user2390246 May 10 '16 at 14:46
0

Software that I have tried in my field: MATLAB, SPSS, WEKA, GAMS, DigSilent, ETAP, PSpice, MS Excel, MS Word, MS PowerPoint, ...

My recommendation: MATLAB

Positive points of MATLAB:

  1. It has both GUI features and coding (of course, they should go for coding).

  2. The community is active and good.

  3. Their documentations is very good.

  4. Their examples are excellent.

These all help learn MATLAb quickly.

What I have done in MATLAB:

Genetic Algorithm, Linear and Nonlinear Optimization, Clustering, Classification, Plotting, Parallel Computing, Neural Networks, ...

Plus, tell them to work with LATEX as well. They will thank you later. Though, they should also be already an expert in MS Word.

When they get into one area in detail, let's say Optimization, then it might be time to move to a new software like GAMS. You will still need MATLAB for plotting your results, ...

Although almost all academic institutes support MATLAB and have a license, industry are fan of more freeware, like Python.

  • This answer adds nothing to the already existing answers. – Mast May 10 '16 at 15:48

Not the answer you're looking for? Browse other questions tagged or ask your own question.