7

I'm interested in best practices for research workflows for quantitative social science. The best book I have encountered on the subject is "The Workflow of Data Analysis Using Stata" by J. Scott Long (2008). As the title suggests, it is largely about working with Stata, but the author begins by discussing general characteristics of a good workflow: accuracy, efficiency, simplicity, standardization, automation, usability, and scalability. (It is also a decade old now, and does not discuss version control software, unit testing, etc.)

What other resources (books, websites, etc.) are helpful for designing research workflows for quantitative social science, both generally and for specific packages (Matlab, Python, R, ArcGIS, Stata, etc.), and what are the strengths and weaknesses of each such resource?

closed as off-topic by cag51, scaaahu, user3209815, vonbrand, J-Kun May 20 '18 at 16:07

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "“Shopping” questions, which seek recommendations or lists of individual universities, academic programs, publishers, journals, research topics, or similar as an answer or seek an assessment or comparison of such, are off-topic here. (See this post on meta for more information.)" – cag51, scaaahu, user3209815, vonbrand, J-Kun
If this question can be reworded to fit the rules in the help center, please edit the question.

  • A group to check out is Data Carpentry; unfortunately, most of what I've seen is "build-your-own-workflow," but that also kind of makes sense when you go beyond just one or two tools. Good luck! – cactus_pardner May 12 '18 at 17:55
  • Can you edit your question to avoid being a "shopping" question, which is off-topic for the site. – Richard Erickson May 15 '18 at 21:48
  • @RichardErickson I could, but my schedule doesn't have room for that for at least the next week. I could delete the question (and my answer) and consider another attempt later. – FalafelPita May 17 '18 at 15:23
  • @FalafelPita What about is this rephrased question: What recommendations do you have for designing a practical research workflow in quantitative social sciences? Please state also which resources and tools you find helpful for different phases of research like planning, organizing, documenting and computing. – non-numeric_argument Jun 22 '18 at 8:57
3

Overview

To my knowledge, only a very few research workflows for specific packages / tools have been published (the R and Stata books listed below).

As the question is about research workflows, I have avoided the many published guides about how to use a particular quantitative analysis application, with the exception of the Data Carpentry website, because it is explicitly designed for social scientists and emphasizes the importance of a reproducible analysis.

I included the section on version control because, in my experience, it is not commonly used by social scientists.

General Resources

  • (pdf) Code and Data for the Social Sciences: A Practitioner’s Guide by Gentzkow and Shapiro (2014)
    • Topics include automation, version control, file organization, data management, abstraction, documentation, project management, and code style. Employs a conversational. Freely available at the link.
    • Strength: Topics include most aspects of a research project. The authors are social scientists who sought better workflows.
    • Weakness: Level of detail in the examples and anecdotes varies considerably across principles.
  • (pdf) The Plain Person's Guide to Plain Text Social Science by Kieran Healy
    • Topics include: text editors (especially emacs) and markdown, version control, automation with makefiles, and using pandoc to generate HTML and Latex for presentation documents. Freely available.
    • Strength: Provides many links to valuable information on particular tools and topics.
    • Weakness: With the exception of the emacs starter kit section, the material is more motivational than instructional.
  • (Article) Best Practices for Scientific Computing published in PLoS Biol in 2014
    • 8 best practices with subparts, including (1) Write program for people, not computers; (6) Optimize software only after it works correctly; and (7) Document design and purpose, not mechanics. Freely available at the link.
    • Strength: Very concise statements of general principles. Contains references to additional information.
    • Weakness: Because the principles are general, they cannot be used as an off-the-shelf guide for specific software packages or tools.
  • (Website) Data Carpentry
    • Strength: Freely available lessons with considerable detail on python, R, OpenRefine, SQL, and spreadsheets.
    • Weakness: Topics are largely restricted to working with data, and do not include other parts of the research workflow (some of which are addressed through the companion Software Carpentry resources)
  • (Website) Software Sustainabilitiy Institute Guides
    • Topics include Data, Project Management, Software Development (including testing), and Repositories and Project Infrastructure, among others
    • Strength: Many short articles on specific topics
    • Weakness: Not a unified work
  • (Book) The Practice of Reproducible Research, edited by Justin Kitzes, Daniel Turek, and Fatma Deniz
    • Strength: Many examples (case studies).
    • Weakness: Very lengthy. Not designed as a quick-reference for practitioners.

Version Control

  • (Book) Version Control By Example, by Erik Sink (2011)
    • Freely available as a website or downloadable pdf.
    • Strength: Explicit examples of how to use Subversion, Mercurial, Git, and Veracity.
    • Weakness: ?
  • (Application) Sourcetree
    • A graphical user interface for version control systems (Git, Mercurial), compatible with GitHub and BitBucket
    • Strength: Good for the less experienced because users don't have to use the command line
    • Weakness: ?

R

  • (Book) Reproducible Research with R and R Studio, by Christopher Gandrud
    • About using R, RStudio, Knitr, and Markdown to perform data management and analysis, document one's work, and create presentation documents in HTML or Latex. Also includes one chapter on using git and github, and one on data gathering using makefiles.
    • Strength: Unified workflow for research from start to finished product.
    • Weakness: ?

Stata

  • (Book)The Workflow of Data Analysis Using Stata by J. Scott Long (2008)
    • Topics include file organization, variable naming, data labeling and documentation, automation and one-click reproducibility, and larger project management organization.
    • Weakness: no discussion of version control software or code testing

Not the answer you're looking for? Browse other questions tagged or ask your own question.