How to avoid gender bias when writing letters of recommendation?

Question

Finding letter writing guidelines that simply say something like "avoid gender bias" is rather easy. Finding letter writing guidelines that explain what this means in more detail is extremely challenging.

What do men and women tend to do differently when writing letters? What do letter writers tend to do when writing about men or women candidates? How can these biases be recognized and mitigated?

This study by Bell, Cole, and Floge shows that male letter writers write differently than female letter writers and that both write differently when recommending men for a position than when recommending women. But that's one study, published in 1992, and it is relatively hard to extract concrete advice on avoiding gender bias when writing recommendation letters from this paper. Is there other work in this area? Can it be distilled into something that's more directly applicable to letter writing?

(Note that there is a somewhat related question about avoiding gender bias when evaluating prospective applicants for an academic position.)

Answer 1

First, here is an answer to "What do letter writers tend to do when writing about men or women candidates?"

One difference I have seen mentioned in the literature but have not seen mentioned in the other answers: use of standout adjectives, words like "superb," "outstanding," and "excellent."

Specifically, in letters for 62 female and 222 male applicants to a medical faculty at a large American medical school in the mid-1990s:

We developed a list of ‘standout adjectives’, namely ‘excellent’, ‘superb’, ‘outstanding’, ‘unique’, ‘exceptional’, and ‘unparalleled’. In tabulating the percentages of letters for female applicants and male applicants that included these terms, we found them to be similar (63% for women and 58% for men). And yet the letters for men read differently. Our gut reaction after reading many of these letters was that the men had been praised more highly with these terms than had the women. Thus we were led to consider frequency. That is, instead of coding mere occurrence of at least one of these terms in a letter, we coded for multiple occurrences. Here we found that the letters for women that had at least one of these terms had an average of 1.5 terms, whereas the letters for men that included at least one had an average of 2.0 such terms. That is, there was repetition of standout adjectives within men’s letters to a greater extent.

Trix, F., & Psenka, C. (2003). Exploring the color of glass: Letters of recommendation for female and male medical faculty. Discourse & Society, 14(2), 191-220.

And in letters written for 235 male and 42 female applicants for either a chemistry or biochemistry faculty position at a large U.S. research university:

In line with Hypothesis 5, results revealed a significant gender difference in how many standout adjectives (e.g. outstanding, unique, and exceptional) the recommender used to describe the candidate, F(1, 278)=3.95, p=.05. Consistent with the notion that implicit biases can influence how letter writers describe female candidates, recommenders described male candidates (M=.70) with significantly more standout adjectives compared to female candidates (M=.60). To address the possibility that this difference could be accounted for by differences in the qualifications of male and female candidates, we conducted an ANCOVA that included number of publications, presentations, fellowships, postdoctoral positions, and number letters of recommendation as covariates. Even after removing variance in standout language due to any and all of these variables, the gender difference remained significant, p=.04. There were no differences between departments in how many standout adjectives candidates' letters included.

The "standout words" from the latter study were: excellen*, superb, outstanding, unique, exceptional, unparalleled, *est, most, wonderful, terrific*, fabulous, magnificent, remarkable, extraordinar*, amazing, supreme*, unmatched

Schmader, T., Whitehead, J., & Wysocki, V. H. (2007). A linguistic comparison of letters of recommendation for male and female chemistry and biochemistry job applicants. Sex Roles, 57(7-8), 509-514.

The latter study also found, however, that "results revealed more similarities than differences in letters written for male and female candidates."

A third study did not find a significant difference in use of standout adjectives by gender of the applicant. In a review of 763 letters for applicants to an otolaryngology/head and neck surgery (OHNS) residency program at Stanford University:

The length of the letters was similar, as were the mean number of "standout" terms.

However, this study did address your other question, "What do men and women tend to do differently when writing letters?":

All 763 letters "recommended" the applicant for OHNS residency. Ninety-one percent of letters were written by men, 68.4% by male otolaryngologists (OTOs), 4.2% by female OTOs, and 33% by OHNS department chairs or division chiefs (100% men). A comparison of female and male letter writers revealed five categories with significant differences: female letter writers were more likely to call an applicant a “team player” (P = .000), “compassionate,” (P = .001) and use strings of adjectives (P = .024). In contrast, they were less likely when compared with male letter writers, to mention an applicant's personal life (P = .003), or write “letters of minimal assurance” (P = .035).

In more detail:

Male letter writers were more likely (28% of letters compared with 10% of letters by female letter writers) to comment on the candidate's personal life, sometimes discussing the candidate's family history (e.g., immigration from another country), major accomplishments (e.g., biking across the United States), or hobbies (e.g., fly fishing). Female letter writers tended to discuss the applicant in terms of his or her professional accomplishments only. They were also more likely to mention that the applicant was a “team player” and “compassionate.” Women were more likely to use a string of adjectives to describe the applicant. Some researchers assert that the use of stringing terms is used to substitute for more substantive language about academic characteristics of the candidate. This did not seem to be the case in this group of letters where, when correlated with the USMLE scores, “stringing of adjectives” was close to positively correlated with higher scores (P = .068). Female letter writers were less likely to write a letter of “minimal assurance.”

Messner, A. H., & Shimahara, E. (2008). Letters of recommendation to an otolaryngology/head and neck surgery residency program: their function and the role of gender. The Laryngoscope, 118(8), 1335-1344.

A more subtle difference that may occur in letters of recommendation involves "sorting" students based on gender stereotypes - for example, speaking more favorably of female students' ability at family medicine:

To gain some insight into this, we performed detailed text analysis of approximately 300 medical student performance evaluations (MSPEs) written for students applying to a competitive diagnostic radiology residency. Results showed subtle differences in the text of MSPEs related to the gender of the author and student suggesting that gender stereotypes and their accompanying expectations and assumptions contribute to the gendered socialization of medical students toward different specialties. For example, factor analysis of word categories in MSPEs found that family medicine, a communal specialty, was positively associated with standout adjectives (eg, excellent, exceptional) only in MSPEs written about female students by female authors. By comparison, male authors rarely mentioned family medicine in writing about male students. In text from female authors writing about male students, family medicine negatively correlated with words indicating ability and insight. These results suggest that, however unintentionally, stereotype-based assumptions that women are communal and men are agentic may lead evaluators to see women as a better fit for communal specialties such as family medicine. Close examination of the text supports this as indicated by the surprise when a male student excelled in family medicine noted by this female author: “[He] really surprised us! [He] is an exceptional student [in family medicine].” The text from another female author appears to express relief that a male student who excelled in the communal setting of family medicine also performed well in the agentic setting of surgery: “Although [he] received highest honors on [his] family medicine rotation, surely [his] finest performance was on surgery… [where he] was outstanding — spoke with families, got consent forms signed, was extremely aggressive…” It is possible that the absence of “family medicine” in text from male authors writing about male students also results from gender alignment (ie, no mention of this communal specialty in letters from agentic authors for agentic students).

Quote from Carnes, M., Bartels, C. M., Kaatz, A., & Kolehmainen, C. (2015). Why is John More Likely to Become Department Chair Than Jennifer?. Transactions of the American Clinical and Climatological Association, 126, 197.

But the study it's from is: Isaac, C., Chertoff, J., Lee, B., & Carnes, M. (2011). Do students’ and authors’ genders affect evaluations? a linguistic analysis of Medical Student Performance Evaluations. Academic Medicine, 86(1), 59.

Answer 2

When I write my letters, I try to consciously avoid the biases that have been reported in the literature. Namely, I try to avoid stressing the effort/emotion/affect of female candidates. For both genders, I try focus on research capability and competence.

You may find these helpful:

• http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2572075/
• http://www.academic.umn.edu/wfc/rec%20letter%20study%202009.pdf
• The graph below can be found in this paper: https://www.ncwit.org/sites/default/files/resources/avoidingunintendedgenderbiaslettersrecommendation.pdf
• the original source is: Exploring the Color of Glass: Letters of Recommendation for Female and Male Medical Faculty doi: 10.1177/0957926503014002277 where the caption reads "Semantic realms following possessives. Rank-ordered within gender sets from equal numbers of letters ‘her training’; ‘his research’."

Answer 3

I am no expert on gender bias, but I want to make comments on this part of the question (which I still think should be split up):

What do men and women tend to do differently when writing letters?

In the study mentioned in the question (which is noted has a small sample size and may be biased), the main observations in relation to this question are:

• men and women were equally likely to discuss teaching, research and collegiality (all positively)
• however, women were less likely to provide support for a positive assessment of research for female candidates
• men wrote about intellect more often

From my personal experience of reading letters for research mathematicians, I'm skeptical of the second point being valid in my field at present because all the non-teaching letters are almost entirely about research, though it's possible men write more about research than women in general. I haven't payed enough attention to the third point to have an opinion.

Then the question asked is

How can these biases be recognized and mitigated?

This maybe is more important for male vs female candidates than male vs female writers. Not that it's not still interesting, but everyone writes in their own way and there is a lot of variation, both among males and also among females. What's a more important question I think is

• How do I write an effective letter for a candidate?

See for instance: this study, which cites the Bell, Cole and Floge paper and finds the biggest difference is in letters for successful candidates versus unsuccessful ones rather than looking at gender differences.

Note: there seems to be a lot more research on the other side of the question about differences between letters for male versus female candidates. You can Google Scholar the study you mentioned and look at citing papers and related papers. See also RoboKaren's answer. However, in my cursory search, I didn't notice any other research addressing male versus female writers.

Answer 4

At http://www.csw.arizona.edu/LORbias you can see a nice one-page poster entitled "Avoiding Gender Bias in Reference Writing". Its suggestions generally echo points already mentioned in other answers, but it's a convenient and concise source.

One common theme is to focus on the person's achievements rather than their effort, potential, or emotional or personal traits.

Answer 5

If you are looking to reduce gender bias in the letters of recommendation you write, perhaps you could try an experiment: pretend that several students that you know well, some male, some female, ask you for a reference. Write a letter for each one. Then black out the name and the pronouns and show them to a friend or colleague and see if the gender can be correctly guessed.