Often I have found that research students who have learned to do experiments have not learned to use software to handle their data effectively. It's not within my power to insert the appropriate training into their prior education. Teaching people to program or use software is not within my area of expertise or a priority for me. I have observed other research supervisors expecting their students to "teach your self to use what I use" but I realize that what I use, while very powerful, may not be best for a beginner. Also, I picked my tools a long time ago so the state of the art may have changed. What should I recommend to my students, and why? I would like for students to rapidly acquire flexible, durable analysis skills.
I would like for students to rapidly acquire flexible, durable analysis skills.
Your criteria are quite stringent! I think you are going to have to compromise at some point along the line. If you want them to acquire the skills rapidly, then they are probably going to have to use menu-based software, which will be limited in its flexibility. The long-term and more flexible solution would be for the students to learn statistical programming, but that of course has a steep learning curve.
In my opinion, R has a lot of advantages. [I imagine you have already come across it, so I may be stating the obvious here and you may have a good reason for ruling it out, but...]
It is free and open source, and therefore once learnt, the skill can be taken anywhere.
It's massively flexible when you take into account all of the add-on packages
Students can "ease" into it using R commander, which gives a menu-based interface but also outputs the corresponding code.
It is popular and therefore very well resourced.
The best compromise that I can think of would be to start the students off using the menu-based R commander package, but encourage them to inspect and customise the code where possible. If you are not able to give training yourself, it would probably be a good idea to arrange for someone else (either in your department, or pay someone external) to give a course. There are lots of good self-learning resources available, but a course ought to speed up the learning process. When they see how powerful the software is, it is likely to encourage them to put in the time and effort to learn to use it well.
Teaching how to program should be a priority in research, in at least Sciences. Doesn't matter what STEM field you are in, almost certainly you will need to deal with data, and using "black box" software only teaches to do whatever the software tells you.
I've seen people give results using standard deviation and mean for non-normal distributions, and its just because they didn't know how to plot their distribution and just used black box software. I've seen people rename file by file a folder with 500 files with data. These are worse than "Attack ships on fire off the shoulder of Orion".
Since nowadays almost all research in STEM is performed using computers, understanding computers is a must.
My recommendation is to make people learn MATLAB if you have access to it, or Python if you don't have access to MATLAB or you are a Open source / free software supporter. Both of this languages are designed to be very high level, and not need "advanced" computer science skills (such as inheritance in OOP, or pointers in C)*. Both of the languages are widely used and there are numerous free online courses to learn, in Coursera, Codeacademy, Udacity, EDx or any other online learning platform.
Learning how to code to the basic point should take less than 2 months, considering that meanwhile the student is also doing other things. And they can save thousands of hours of tedious work.
Let me repeat the key message: We need researchers with programming skills. Its incredibly important skill to be able to perform research in the XXI century.
While this answer mainly focuses in STEM, basic programming in other fields that use statistics is also useful.
*Of course, knowing about that helps.
We're talking about students here, not currently practicing researchers, and so my comment is really made with respect to future trends rather than the current state of play which I believe the other answers address.
I believe that in the future, more and more people will be expected to know how to program if they are going to do any kind of data analysis. Perhaps not on the more theoretical side, but since you said your students are doing practical work, I will assume that is not an issue. Tools like R and Matlab are good places to start if you are unfamiliar with programming and want to get something done right now; but honestly, since the barrier to entry for programming in fully-fledge generic programming languages is so low these days (and expected to get lower), I see no reason not to point students in the direction of a full programming language and the modules they might want for doing statistical analyses that are relevant to their field.
Whilst R and Matlab are fine choices, personally, I would introduce my students to something like Python, and the excellent modules that are avalible to do all the data analysis that can be done in R/Matlab that exist in the Python ecosystem. Python has a very gradual learning curve at the beginner end of the spectrum, while at the other end advanced programmers can write code thats just as fast as C if they take advantage of the newer, optimized interpreters. These 2 pros, plus the plethora of modules for doing any kind of analysis/plotting R or Matlab can do, is what has made Python the defacto language of choice in my field (Bioinformatics), and likely a powerful tool under your student's belts going forward with whatever they decide to pursue in life.
Of course, there are other languages out there, such as Java, Julia, Rust, etc - however I would rather teach those as second or third languages to learn, once you have a strong foundation in Python.
For the record, i'm not saying "teach them python", i'm saying just make them aware of it's existence.
Matlab/Octave is appropriate for physics (experimental)
Matlab is very well documented. and get all the graphs done you need as a physicist. It will allow your students to focus on the physics problem instad of hunting bugs or documentation.
Be sure that your university includes the statistics toolbox in the license.
Often I have found that research students who have learned to do experiments have not learned to use software to handle their data effectively
What kind of data handling and what kind of software program you're referring to? Is it programming language that analyze experimental data (R, Python/C++) or software (like SPSSS/Minitab, Matlab/Octave, Atlas.ti/VUE etc)?
R, C++ would be have a very steep learning curve. Even Python would not an easy entry for student who have no basic programming in their undergraduate years (based on my personal experience, it's not the syntax itself, but the early step on choosing and install IDE, configure module etc but maybe that just me).
This may sound cliche but for not starting flame wars about which software/program is the best, it all depends on the needs and background of the users. For example, those who has background in statistics would like to use R (R can do lot more than statistics, I know) but for biology students who do research in fermentation, maybe Minitab is enough. Same thing with Python/Ruby/Julia or C++/Fortran or Word/LaTeX etc.
Ask them what they learned in their undergraduate years and see if they can fully maximize it in their research. For me, the first step is not to learn fancy software but to have a clear understanding how to do research and a good research practice (research methodology, workflow, raw data management etc) and only then introduce software/program to help them.
The specific platform is going to be discipline specific so what will work for my students may not be best for your students.
However, if you teach them a proper digital-data-analysis workflow, then they will be able to transfer good habits to whatever specific analysis tool they are using.
For my students, I emphasize:
1) proper data archiving - each data set contains only data stored as plain text
2) proper metadata - each data set has accompanying detailed metadata
3) using code for data analysis - all data manipulations are completed with reproducible code and the original data are never altered
4) version control - we use git but that's just my preference
Software that I have tried in my field: MATLAB, SPSS, WEKA, GAMS, DigSilent, ETAP, PSpice, MS Excel, MS Word, MS PowerPoint, ...
My recommendation: MATLAB
Positive points of MATLAB:
It has both GUI features and coding (of course, they should go for coding).
The community is active and good.
Their documentations is very good.
Their examples are excellent.
These all help learn MATLAb quickly.
What I have done in MATLAB:
Genetic Algorithm, Linear and Nonlinear Optimization, Clustering, Classification, Plotting, Parallel Computing, Neural Networks, ...
Plus, tell them to work with LATEX as well. They will thank you later. Though, they should also be already an expert in MS Word.
When they get into one area in detail, let's say Optimization, then it might be time to move to a new software like GAMS. You will still need MATLAB for plotting your results, ...
Although almost all academic institutes support MATLAB and have a license, industry are fan of more freeware, like Python.