11

Common sense dictates that any and all work/research/drafts/etc that we do in our journal-research and PhD ought to be backed up and backed up again. Most of the time, this is constrained to laptop/desktop, USB memory and perhaps a copy in your own email - one disaster could wipe all that out (almost happened to my MSc research - got caught up in a severe natural disaster).

My question is, is there a more secure and 'safe' means to save/backup our PhD and/or journal-research work? Do universities offer such services, or is it up to individual researchers to find their own solutions?

I should note that I have no illusions of any system being completely 100% safe and secure, but are looking at ways that are more secure than the 'traditional' methods listed before in the first paragraph (2nd sentence) of this question.

  • 9
    I'm not sure to see if it's very relevant to academia, and not just a backup problem. In that case, it might be a better fit for superuser.com. – user102 Jun 28 '13 at 15:57
  • 10
    Perhaps not the answer you were looking for, but publishing your work is in some sense the "ultimate backup". This is why I try to put every relevant piece of research into the manuscript (much as supplemental material) and submit as much data as possible to public databases. That way you only need to backup what you are currently working on. Of course this also helps people who want to further explore my published research. – Bitwise Jun 28 '13 at 19:09
  • 5
    @Damien: We have had some discussions in the past about "boat programming" questions: meta.academia.stackexchange.com/q/459/102. I'm not saying your question is not relevant, I'm just wondering whether you wouldn't get better and more accurate answers at another SE, since it does not seem to be particularly specific to Academia. – user102 Jun 28 '13 at 22:24
  • 1
    Further detail can be found here, but the idea is to decide if a question is very specific to academics, or if it could apply to many other categories. For instance, to exaggerate, you could ask "what is the best car for academics?", using the argument that some academics do use a car to go to work. Such a question would clearly be off-topic. In your case, it's not clear whether it's such a situation or not, and that's why I didn't close the question, but merely suggested that it might not be very specific. – user102 Jun 28 '13 at 22:45
  • 2
    Not an answer for you, but my lab have automated daily backups of our lab computers. I would expect more experimental labs, using lot of big data, to have such services, but it seems to be rare. – Benoît Kloeckner Jun 29 '13 at 11:18
11

Backups, short-term and long-term storage of research material is actually something we have to address whenever we apply for a IRB for a research study or an IRB exemption. We have to spell everything out in great detail including when we are going to destroy or get rid of the acquired research materials (since they usually contain personal identifiers like name, address, email etc.)

This is our approach:

  1. We have our own encrypted lab server on a RAID configuration. Access to the server is restricted to myself and my adviser.

  2. We store generalized, public versions of our scripts on github. The previous answer has very good specifics on github so I will not go into the details here.

  3. Periodically, we backup our lab server to another externally hosted server (not physically located in the US). This backup script is encrypted and password protected.

  4. Physical interview questionnaires, survey instruments, interview transcripts are stored in a locked file cabinet with access to my adviser and myself in our lab. Entry to our lab is through keycard only.

  5. Interview recordings, videos and audios are also stored and backed-up according to the procedures in 1 and [3].

This approach seems to have worked so far.

| improve this answer | |
  • Having only two people knowing the password makes the Bus factor very high (plus the more secure it is, it is easier to loose access to it). So there is often a trade off (surely, for some confidential data it may be better to loose access to it than to disseminate). – Piotr Migdal Jun 29 '13 at 9:20
  • Sure, but we don't develop commercial software. We write scripts which implement various algorithms which we derive mathematically. Said scripts are uploaded on github which everyone can see. Our collected data is protected. – Shion Jun 29 '13 at 12:41
6

For most cases, using git or hg is too much for research that don' t include much code.

For a concise answer:

Put everything in a web service like dropbox that provides function of version control. Dropbox is easy and straightforward to use and don' t bother you with complex version control commands, while it provides file version system for last 30 days. The free space is usually enough for current work.

For code, you can put at bitbucket.org, since it offers nice unlimited plan for academic use, which offers you private repos and unlimited team member. Of course you can open source your code at github.

| improve this answer | |
  • 1
    Note that with a cloud service (except mega...), you should expect the NSA and other agencies to have have access to your research. This is usually not a problem, but in some cases that can be a relevant issue. – Benoît Kloeckner Jun 29 '13 at 11:16
  • In the case with NSA concerns, you can setup your own file syncing server inside your lab using some open source solution like seafile. – xgdgsc Jun 30 '13 at 3:09
  • In my institute it would be a concern (even without NSA). So we use an internal git server with access rights for collaborators. IMHO git is also good for research that includes writing text, not only source code. – cbeleites unhappy with SX Jun 30 '13 at 18:10
  • AAAAAAAARGH be careful of sync services like Dropbox that don't version! The horror scenario (and it's happened to people!) is that you fumblefingeredly delete something, Dropbox syncs the deletion, and then the file exists in NEITHER PLACE. – D.Salo Dec 23 '15 at 3:23
  • 1
    @D.Salo I believe Dropbox keeps a copy you can restore for some time. As long as you realize you accidentally deleted something, you should be alright. – mrm Dec 23 '15 at 7:48
3

This is not really a research-related question at all, but I can tell you what I do. I put everything (excluding large data sets) under distributed version control (Mercurial) and push to remote repositories frequently. NB: You can easily automate the pushing if you desire; this has positive and negative aspects.

Of course, the two leading DVCS are Git and Mercurial, so either of these would work. Free hosting for Git is available at Github and Bitbucket, free hosting for Mercurial is available on Bitbucket. There are many other places you can put your repositories, of course, including remote servers your organization owns.

Note: large data sets require special handling, so I'm not sure about that, but they are often not original work, and can be retrieved from other places if necessary.

I strongly recommend doing this in conjunction with some sane method for saving "scratch" work (i.e. work in progress that is not yet suitable for a fully formed commit), so you don't have significant unsaved work lying on your hard disk. The important point here is to have a workflow by which you can sanely push commits at frequent intervals - I try to make commits every 1/2 hr or less. I use Mercurial queues. There is also a brand new thing called the Evolve Extension, but I have not tried that yet. Git has similar software available. See for example, the Stack Overflow question git equivalent to hg mq? and What's the Git approach to publish a patch queue?. NB: If you do use a patch queue, you need of course to version it, otherwise you can't push it.

These methods can appear complex if you are not used to version control. However, I can attest this approach works well. My workstation died three times in the last 3 1/2 years, and each time I was able to switch to a different machine and carry on working without interruption, without access to the machine I had just been using.

See also Why use version control systems for writing a paper, particularly user244795's answer.

| improve this answer | |
2

To add a slightly less "public" view (my institute is a non-university research institute which would be counted either as academic or industrial depending on whom you ask). We mainly work with original data we measure ourselves. The major concern of our administration seems to be that original data does not inadvertently end up becoming public, destroyed/lost data seems to be less of a concern (but there are backed up servers, see below).

  • Default policy is that data and code (and paper drafts) stay inside the institute (though we may take them home, as working from home office is a valid and frequently used option) and within the collaborating groups, respectively.

  • Basically this has not only the obvious consequence of no google docs, no public github or the like (unless specified for the particular project) but also e.g. no private github plan.

  • Instead of private github plan (or the like) we have our own git server. The main purpose of the server is not backup (it is located in the main server room, so e.g. a fire killing the file server would most likely kill the git server as well), but collaboration. However, having collaboration with a DCVS automatically means backups that are distributed as far away as the collaborators are.

  • OTOH, our computer services provides a file server which is automatically backed up.

  • Once a publication is out, original data, source code and the paper are backed up in a publication data base as well.

  • Due to home office, I have an external hard disk that I carry hence and forth (particularly for data that is too big to be conveniently exchanged via internet with the git server). Which means that unless the big disaster happens when the external hard disk, my laptop, my desktop, and the file server with the backups are all destroyed (what is the probability that I'm well and up to work after such an event?), there will be some copies surviving.

  • At a university where I worked before, the central computer administration did backup services for specified computers of the institutes. We once needed that after a fire in our institute's server room.

  • I may add that from my perspective of a scientist who is also doing measurements on physical (in fact, biological) samples, I've experienced much more trouble due to physically destroyed or spoiled samples than with computer failures.
    Sure, we have our share of instrument computers dying (big issue, as it can be difficult to find parts that are compatible particularly with older instrument hardware - but no data loss issue: measurements are usually immediately transferred to file server, office computers, external hard disk). Office computers seldom die without warning symptoms beforehand. There also was that fire in the server room (university computation center back-up did work).
    In the same time, we've had 3 failures of deep-freezers where our tissue samples are stored. Plus other accidents (tissues thawed by inexperienced student, lots of tissues never frozen correctly by collaborating surgeons, ...).

    So even without a "proper" data backup plan like automatic backup of all computers, physical loss of delicate samples seems to be the more pressing problem than loss of data.

| improve this answer | |
2

The classic digital-preservationist mnemonic for this is "3-2-1."

  • THREE copies of everything, on at least
  • TWO different types of storage medium, with
  • ONE offsite.

The storage-medium thing is so that you don't get bitten by (e.g.) a bad batch of one manufacturer's hard drives (cough cough Seagate) or a general weakness in a particular medium (such as USB drives' propensity to get lost or break). Offsite is so that a building fire or earthquake or tornado doesn't do you in.

The lowest-stress way to handle this is typically a backup-designated hard drive in your office with automated backup scheduling, plus cloud storage -- though be wary of cloud-storage that syncs but does not keep deleted files (DROPBOX), because you might delete a file, have the deletion synced, realize you NEED the file, and then be unable to get it back from the cloud.

If your data need heightened security and your campus doesn't have any suitable storage offerings (unfortunately common), look for a "zero-knowledge" cloud-storage provider such as SpiderOak or Tresorit. DO NOT half-ass your security measures! Leaking personally identifiable information has ended research careers (look up "Yankaskas" for the horror story I use in my classes).

| improve this answer | |
  • Dropbox should keep a copy you can restore for some time. – mrm Dec 23 '15 at 7:50
1

Because no-one's mentioned it yet, I use and love CrashPlan (http://crashplan.com). It's cheap, everything is backed up and encrypted to "the cloud" and it's always running. A nice complement to full disk cloning and git/mercurial.

| improve this answer | |
  • I have never heard of this, is it user friendly? – user7130 Jun 30 '13 at 0:34
  • 1
    Very - just runs in the background silently doing its job. It's an easy and cheap and always working offsite backup. (Note: this might sound spammy, but I'm just a happy customer) – Simon Jul 2 '13 at 11:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy