Resources for quantitative social science research workflow [closed]

Question

I'm interested in best practices for research workflows for quantitative social science. The best book I have encountered on the subject is "The Workflow of Data Analysis Using Stata" by J. Scott Long (2008). As the title suggests, it is largely about working with Stata, but the author begins by discussing general characteristics of a good workflow: accuracy, efficiency, simplicity, standardization, automation, usability, and scalability. (It is also a decade old now, and does not discuss version control software, unit testing, etc.)

What other resources (books, websites, etc.) are helpful for designing research workflows for quantitative social science, both generally and for specific packages (Matlab, Python, R, ArcGIS, Stata, etc.), and what are the strengths and weaknesses of each such resource?

Answer 1

Overview

To my knowledge, only a very few research workflows for specific packages / tools have been published (the R and Stata books listed below).

As the question is about research workflows, I have avoided the many published guides about how to use a particular quantitative analysis application, with the exception of the Data Carpentry website, because it is explicitly designed for social scientists and emphasizes the importance of a reproducible analysis.

I included the section on version control because, in my experience, it is not commonly used by social scientists.

General Resources

• (pdf) Code and Data for the Social Sciences: A Practitioner’s Guide by Gentzkow and Shapiro (2014)
  • Topics include automation, version control, file organization, data management, abstraction, documentation, project management, and code style. Employs a conversational. Freely available at the link.
  • Strength: Topics include most aspects of a research project. The authors are social scientists who sought better workflows.
  • Weakness: Level of detail in the examples and anecdotes varies considerably across principles.
• (pdf) The Plain Person's Guide to Plain Text Social Science by Kieran Healy
  • Topics include: text editors (especially emacs) and markdown, version control, automation with makefiles, and using pandoc to generate HTML and Latex for presentation documents. Freely available.
  • Strength: Provides many links to valuable information on particular tools and topics.
  • Weakness: With the exception of the emacs starter kit section, the material is more motivational than instructional.
• (Article) Best Practices for Scientific Computing published in PLoS Biol in 2014
  • 8 best practices with subparts, including (1) Write program for people, not computers; (6) Optimize software only after it works correctly; and (7) Document design and purpose, not mechanics. Freely available at the link.
  • Strength: Very concise statements of general principles. Contains references to additional information.
  • Weakness: Because the principles are general, they cannot be used as an off-the-shelf guide for specific software packages or tools.
• (Website) Data Carpentry
  • Strength: Freely available lessons with considerable detail on python, R, OpenRefine, SQL, and spreadsheets.
  • Weakness: Topics are largely restricted to working with data, and do not include other parts of the research workflow (some of which are addressed through the companion Software Carpentry resources)
• (Website) Software Sustainabilitiy Institute Guides
  • Topics include Data, Project Management, Software Development (including testing), and Repositories and Project Infrastructure, among others
  • Strength: Many short articles on specific topics
  • Weakness: Not a unified work
• (Book) The Practice of Reproducible Research, edited by Justin Kitzes, Daniel Turek, and Fatma Deniz
  • Strength: Many examples (case studies).
  • Weakness: Very lengthy. Not designed as a quick-reference for practitioners.

Version Control

• (Book) Version Control By Example, by Erik Sink (2011)
  • Freely available as a website or downloadable pdf.
  • Strength: Explicit examples of how to use Subversion, Mercurial, Git, and Veracity.
  • Weakness: ?
• (Application) Sourcetree
  • A graphical user interface for version control systems (Git, Mercurial), compatible with GitHub and BitBucket
  • Strength: Good for the less experienced because users don't have to use the command line
  • Weakness: ?

R

• (Book) Reproducible Research with R and R Studio, by Christopher Gandrud
  • About using R, RStudio, Knitr, and Markdown to perform data management and analysis, document one's work, and create presentation documents in HTML or Latex. Also includes one chapter on using git and github, and one on data gathering using makefiles.
  • Strength: Unified workflow for research from start to finished product.
  • Weakness: ?

Stata

• (Book)The Workflow of Data Analysis Using Stata by J. Scott Long (2008)
  • Topics include file organization, variable naming, data labeling and documentation, automation and one-click reproducibility, and larger project management organization.
  • Weakness: no discussion of version control software or code testing