Home → Magazine Archive → November 2014 (Vol. 57, No. 11) → The Data on Diversity → Full Text

The Data on Diversity

By Beryl Nelson

Communications of the ACM, Vol. 57 No. 11, Pages 86-95

[article image]

Save PDF

People working together can achieve more than they can alone; this is a fundamental principle upon which organizations are founded. Social scientists have shown that teams and organizations whose members are heterogeneous in meaningful ways, for example, in skill set, education, work experiences, perspectives on a problem, cultural orientation, and so forth, have a higher potential for innovation than teams whose members are homogeneous. These findings are not without controversy, yet the implications for the computing industry are profound, given the relative homogeneity of the field along a few important dimensions. Take, for example, the composition of degrees awarded in computer science, computer engineering, and informatics in 2012 at research institutions in the U.S.

Back to Top

Key Insights


  • 13.3% of BS degrees, 28.7% of MS degrees, and 19.2% of Ph.D.'s were awarded to female candidates, down from a high of 37% of BS degrees in computer science in 1986.
  • 5.3% of BS degrees were awarded to African American candidates, as were 2.7% of MS degrees, and 2% of Ph.D.'s.

Among computing professionals, about 20% of CS faculty in U.S. universities are women, and 1.6% are African American.50 Similar numbers exist in industry.

Diversity, bias, and stereotypes have traditionally been discussed in very relativistic terms: surveys of whether people thought there was bias, and so on. In recent years, imaginative researchers have developed ways to gather quantitative data about the benefits of, as well as the challenges to, having a diverse workforce. This article explores the benefits that diversity can bring to teams, and the cognitive factors—namely, stereotypes based on social group membership—that keep us from achieving optimal levels of diversity.

Back to Top

Benefits of Diversity

Diverse teams are more effective: they produce better financial results and better results in innovation. These results show that having a diverse organization is a business imperative.

Financial results. Organizations that include a high percentage of women in senior positions show better financial results.

Companies in the top quartile for women in the executive committee from 2007–2009 had 41% greater return on equity, and 56% greater earnings before interest and taxes than companies with no women in the executive committee, for companies within the same industrial sector (see Figure 1).15

Financial results for companies with at least three women serving on the board of directors are better: in 2007, return on equity was 16.7%, as opposed to an average 11.5%; return on sales was 16.8%, as opposed to an average 11.5%; return on invested capital was 10%, as opposed to an average 6.2% (see Figure 2).26

This is also true in other geographies. Profits of Indian companies headed by women grew 56% within five years, but grew even faster, at the rate of 64%, within three years.33 The BSE-30 companies posted a growth rate of 27% and 23%, respectively, during the same period.

Similar results have been found for race: organizations with greater racial diversity were associated with greater sales revenue, a larger number of customers, greater market share, and greater profits. In the same study, greater gender diversity was also found to be associated with better results in sales revenue, number of customers, and profitability.23

These studies are correlational: causation can be inferred, but not proven. Researchers have posited that diversity may actually be one causal determinant of firm performance due to the increased innovation that occurs when organizations are diverse. The next section reviews this evidence.

Innovation. Diversity has been shown to create a cognitive and social environment that is a positive indicator for innovation and a negative indicator for routine tasks. These dynamics may have real-world consequences for scholarship in our field.

In a study of collective intelligence and creativity, researchers gave subjects aged 18 to 60 standard intelligence tests and assigned them randomly to teams. Each team of 3–5 people was asked to complete several tasks, including brainstorming, decision making, and visual puzzles, and to solve one complex problem that was too difficult for one brilliant individual to solve: that is, a team was required. Teams were given intelligence scores based on their performance.

The only predictor of team collective intelligence was whether there were women on the team. Note that all the high-scoring groups are close to 50% women; some of the low-scoring teams are also near 50%, but the groups with little gender mix did not score as highly. This was a surprise result to the researchers. With more investigation, it was found the difference was having the social skills that made it possible to use the contributions of all the team members, and these correlate more with women than with men. Figure 3 illustrates the relationship of team composition to success.48,49

Patent authorship is another area that shows a benefit of diversity. More than 90% of all computer technology patents issued in the U.S. since 1980 have been granted to men only. Yet mixed gender patents are cited 26% to 42% more than any single gender patent.4 An update to this report in 2012 showed that mixed gender patents typically have a large number of authors. The higher citation rate (30% to 40% more in the 2012 update) is associated with higher numbers of authors. The reasons for this are not well understood.

These results show a correlation between diverse organizational composition, financial success, and innovation. While there is not a clear causal relationship shown between diversity and success, the results have been shown with varying methodologies and in varying geographies, to a degree that demands attention.

Back to Top

Challenges Faced by Diverse Teams

Unfortunately, it is not easy to make diverse teams effective.a There are a number of forces that work against the desired effect: having the entire team productive. There can be potential negative effects of any of the following:

  • unconscious bias,
  • stereotype threat,
  • exclusion from critical social networks,
  • lack of role models, and
  • unaware managers.

The following sections address primarily unconscious bias and sterotype threat.

Unconscious bias. One of the factors that both limits the diversity present in a team or organization, and that inhibits the potential success of diverse teams, is the unconscious bias, that is, stereotypes, that we hold toward people based on the social groups to which they belong. Stereotypes are, simply, a constellation of traits, characteristics, skills, and values that we ascribe to members of social groups, such as gender, race, age, religion, nationality, and others. These are learned through cultural messages and stories, comments from family and friends, portrayals in the media, and so forth. These stereotypes, despite our best intentions, can bias our impressions of, and affect our actions toward, others in our environment. The stereotypes especially relevant in work situations include those characteristics that are visible, such as sex, race, weight, and age; but also those not visible but relatively easy to discern, such as educational background and nationality.

Project Implicit at Harvard hosts an online test of implicit associations: the user's implicit association between two concepts is measured via user response time. There are reports for many associations, and the results are stunning.5

  • Almost everyone has measurable biases (for example, 70%–80% have biases against women in technology, or preferring white to African American, or preferring young people).
  • Almost no one reports such biases (for example, 15% report a preference for white people).
  • Even the people who are the subject of a bias may have that bias. For example, I tested as moderately biased against women in science and technology, and this is totally against my self-interest.

The book Blindspot5 explains the development of the implicit association test and its results, exploring the reasons for differences between unconscious perception and our assumptions. For example, it has been found that age is one of the strongest biases tested; this is true even among the elderly, and in societies in Asia that traditionally have valued the wisdom of age and experience.5 Blindspot also considers the evidence for any predictive link between measured biases and outcomes or behaviors; results are not yet conclusive in this area.

In individual cases, bias can be very difficult to verify. In aggregate, however, bias and other forms of unconscious decision-making are readily apparent:

  • About 58% of CEOs of Fortune 500 companies are taller than 6ft. (about 183cm), and almost a third are taller than 6ft. 2in. (about 188cm). In the population in general, about 14.5% are taller than 6ft., and 3.9% taller than 6ft. 2in.17 Most people would not believe height predicts competence, and yet, these choices are frequently made. In fact, height is strongly correlated with career success.
  • In a laboratory situation, applicants were seen in the waiting room either alone, or sitting next to another applicant. Applicants who were seen sitting next to an overweight applicant were less likely to be hired than an applicant sitting alone or next to an average weight applicant, and regardless of their own weight.46
  • In an examination of data on wages of a given population over the course of five years, it was observed that there is a correlation between weight and income: for women, this is a negative correlation; for men, a positive correlation, excluding obesity.27
  • A disturbing finding is that African Americans and other minorities receive less effective healthcare interventions and outcomes, even when controlling for socioeconomic class and insurance coverage.5
  • At a coffee station at a company in Northern England, the sign for the amount to pay for milk (to put in tea or coffee, on an honor system) had a photo of flowers or eyes on alternate weeks. The amount of money received was about double in the weeks with eyes over the weeks with flowers. And if the eyes were looking directly out, the amount received was much higher; see Figure 4. Although their behavior was apparently heavily influenced by the photos, at the end of the trial, no one remembered the photos changing.7 Clearly, there was unconscious decision making in each of these cases, but the data alone does not give tools to overcome bias.

Science faculty at respected U.S. research institutions show measurable bias toward male students: the faculty were given applications for lab manager that varied only in the gender of the name. Male students were rated significantly more competent, and were given higher average salaries and more mentoring opportunities than women students. Importantly, even the female faculty showed this bias.32 Male faculty offered $30,520.83 to male students on average, and $27,111.11 to female students. Female faculty offered $29,333.33 to male students on average, and $25,000.00 to female students.

Shankar Vedantam explains the bias illustrated in these examples by saying that we create a set of mental shortcuts, or heuristics, for the situations we experience every day, and that we actively work against those biases for the most part, to the extent we think the bias is not "good." For example, you see very small children, as well as people with dementia, expressing strong bias; or politicians may make a slip of the tongue when they are tired.46

Until this point, I have been describing the stereotyping process as a negative force for individual and team functioning. However, this process actually stems from an adaptive, often functional psychological process. Mental heuristics and cognitive shortcuts enable us to process information without conscious deliberation: they fill in the informational gaps we often experience when making decisions. In other words, habits of mind help us to save brain power for more difficult tasks. Joseph Pieper's classic Leisure, the Basic of Culture35 sets out this theory well. In the 1980s, people thought we could create expert systems by interviewing experts like brain surgeons or oil exploration specialists, and creating a rule chaining prolog environment that would recreate their decision-making ability. The problem was the experts did not know how they knew what they knew. That is, experts are creating associations between disparate experiences and pieces of knowledge, using the subconscious brain. In software development, we see this in our most skillful engineers: they can debug a complex problem, but they will probably not be able to specify a set of rules that would work as well as consulting the expert.

What it does say, however, is that we should have better knowledge of our own biases and unconscious decision-making.

Stereotype threat. In Whistling Vivaldi,47 Claude Steele published a history of research about stereotype effects and identity threat. The original problem he tried to solve was, why were college entrance scores not predictive of college success for African Americans? Steele asserts we each have multiple identities (such as I am a woman; I am an MIT alumna; I am American); and that we respond to the identity under threat. He says, "Being threatened because we have a given characteristic is what makes us most aware of being a particular type of person." In Steele's terminology, each of these identities can have what he calls "identity contingencies;" each of these contingencies can be positive, neutral, or negative in a given social situation.47

Based on research into this and other areas, including the success of girls and women in science and math, it has been established that when someone is confronted with a situation that is consistent with a stereotype, and that stereotype places his or her identity with a negative contingency, and if the person cares about this, then performance suffers. For example:

  • Girls perform as well as boys in math tests when there is no stereotype threat, but significantly worse when there is high stereotype threat.39
  • It starts young: Girls aged 5–7 show worse performance on a math test if they first color a picture of a girl holding a doll, compared to coloring a picture with an Asian child eating with chopsticks, or a landscape.2
  • Women do well at math tasks if told that women do well at these sorts of exercises, or if they read about successful women before the test.30
  • White men perform worse in math when reminded of the Asian math stereotype.3
  • Black men perform better than white in athletics ability tests.42
  • But worse if the problem is presented as a problem of "sports strategic intelligence."42
  • Note that the stereotype does not have to be explicitly mentioned for the effect to be felt.40

People in general do not report they are under stereotype threat; they say they do not feel any stress.47 But there are physiological effects that can be measured: blood pressure, sweat; and which correlate with performance.14

Why is this important? We want to distinguish between people who can do work but are stressed, and people who cannot do the work. Moreover, these effects are continuous: it does not end at the job interview. An ideal environment allows everyone to perform at their maximal level, but stereotype threat interferes with this.

Stereotype threat can be mitigated in a number of ways. Note that developing trust is essential, and several of these suggestions help:

  • A credible statement that "this test has been shown to not be subject to the stereotype threat" or "this test is a study of problem solving, not diagnostic of individual ability."39
  • Remind the subject of a positive stereotype: "You are a student at Stanford," or "You are Asian."36
  • Critical mass is defined by Steele as "the point at which there are enough minorities in a setting that other minorities no longer feel uncomfortable there because they are minorities." An exact number needed is difficult to define, but in orchestras, 40% women was seen to be the critical mass: "the point at which men and women alike began to report more satisfying experiences."47 In the U.S. Supreme Court, adding a second woman made a difference.44 In another experiment, a job brochure had photographs of employees; African American applicants start to feel safe in applying once they saw 33% of the faces being minority, whereas white applicants felt safe at all levels of minorities (33% was the highest used in the experiment).37 When women take a math test in a group of three women, they do better than women in groups with two women and one man; and women in those groups do better than women in groups with one woman and two men.25
  • Offering a credible narrative: It helps to hear the experiences of people who have been through similar experiences and who have overcome the situation.47 This includes an expandable view of intelligence (that is, you can learn), and self-affirmation, as explained in the next point:
  • Spending 15 minutes writing an affirmation of one's positive values has been shown to have an enduring positive effect on classroom performance.12
  • Improving critical feedback: neutral feedback, or encouragement, does not work to reduce the stress of identity threat. It works to say, "I have high standards, I think you can meet them, and we are going to work together to make this happen." The difference is engagement.13
  • Communities of support: a key difference between African American students and Asian students at U.S. colleges turned out to be that the Asian students would get together, share their experiences, and study together. People who feel isolated sometimes refuse help, as they think that it will confirm the stereotype. The African American students did not tend to form study groups naturally; when these communities were introduced, their performance increased dramatically.45
  • Fostering intergroup conversations as learning opportunities. These conversations are difficult, as people are afraid of appearing to be biased or ill informed, so they often steer away from the situation. Subconsciously, they pull their chairs further apart. When a facilitator starts a conversation by saying that tensions are natural, but the conversation should be treated as a learning experience, that signals to people that differences can be learned. People pull their chairs closer together. This does not happen when the facilitator says the participants will not be judged, nor when the facilitator says all points of view are welcome.18

Consequences of stereotyping. As mentioned earlier, there are a number of other forces that can hinder the ability of diverse teams to function optimally.

Stereotypes make us feel as though we have useful information about people's strengths, weaknesses, and personal characteristics. But they operate in a more prescriptive way as well: They shape our expectations of what people should be doing, especially at work. Thus, women are expected to be nurturing and collaborative at work, in accord with their stereotyped strengths. When they deviate from that, they incur penalties. Men are not expected to be altruistic, but if they are, then they are given credit for giving assistance. This was demonstrated in an experimental setting in which a person was asked for help with a technical task: men were given an increased performance rating if they gave assistance, whereas a woman who assisted was not; her performance rating remained at the base level. If a man refused to assist, then the rating of his work was not changed from the norm; if a woman refused to assist, her performance rating decreased.21

Most people would not believe that height predicts competence, and yet, these choices are frequently made. In fact, height is strongly correlated with career success.

If a man and a woman perform a male-stereotyped task together, the majority of people will attribute the success of that task to the man, unless

  • the contributions of the woman are specifically attributed, or
  • there is information about the way the task was structured, or
  • there is some clear example of prior competence by the woman.22

There are differences in the communication styles considered acceptable in men and women leaders: women have a very narrow band of acceptable behaviors. In general, women are given less time to speak, are more likely to be interrupted, and if they do interrupt someone, that is most likely to be another woman.43 As a consequence, women are less likely to be able to hold the floor in meetings: an important quality in a leader.

Career advancement strategies that work for men do not always work for women. When comparing only women who have not taken career breaks for family with men, there are still significant differences in achievement levels.10 The study on the Myth of the Ideal Worker says, "Women benefit most by making their achievements known. Men benefit most by scanning for external opportunities and blurring work-life boundaries. Both benefit by gaining access to powerful others." The report also says, "changing jobs accelerated compensation growth for men, but slowed it for women." It has been shown that sponsorship, in which a sponsor actively promotes and takes risks for the sponsee, is more effective for women than mentorship, in which the mentor merely gives advice.24

There are other career differences observable between men and women as well: women are more than 10% less likely than men to change jobs for a raise in salary or for a promotion; but they are almost 10% more likely than men to change jobs because of a bad manager (see Figure 5).9 The numbers who leave jobs to take care of family are very small for both men and women. In Japan as well, significantly fewer women report they have off-ramped because of childcare-related reasons (32%) than those who have off-ramped because they feel stymied and stalled at work (49%).11 One interpretation: It is difficult to balance both working and parenting, and one is not likely to want to stay unless the work is rewarding. Moreover, career priorities differ: A one-size-fits-all management structure will not work well for a diverse organization.

This theory is supported by a study of why women stay in engineering: people who choose to stay are engaged and basically hopeful, and are proactive about the problems they see.8

Even a small bias can result in a large difference in the representation of minorities at the top levels of a company. A simulation of promotions was performed with only 1% of bias in ratings of women, and a posited eight levels for promotion. The simulation starts with 50% women at each level, and ends when the entire organization has been replaced with new employees. At the end of the simulation, at the lowest level of the organization there are 53% women, and the top levels of an organization go to only 35% women.29

Transgender studies show some fundamental differences in the ways that society treats the same person as either a man or a woman. These studies are particularly interesting in that the same person experiences life as both genders. The studies consistently show that men who change to women have an average lower salary after the change, and women who change to men have a higher salary, as well as finding everyday life situations easier.38 A very powerful example is the history of two biologists at Stanford University, Ben Barres (a man, formerly a woman) and Joan Roughgarden (a woman, formerly a man).46

Ben Barres has said, "When it comes to bias, it seems that the desire to believe in a meritocracy is so powerful that until a person has experienced sufficient career-harming bias themselves they simply do not believe it exists ... By far, the main difference that I have noticed is that people who don't know I am transgendered treat me with much more respect: I can even complete a whole sentence without being interrupted by a man." Joan Roughgarden, on the other hand, has said that as a woman, "You get interrupted when you are talking, you can't command attention, but above all you can't frame the issues."

Ben Barres also wrote a strong commentary in Nature6 refuting Lawrence Summers' speech, in which Summers implied that women inherently have less aptitude for science. He cited studies showing that many selection processes set an extremely high bar for competence for women and minorities: 2.5 times in the case of a research grant proposal. And yet, most people have a strong desire to believe the world is fair, so there is widespread belief that little discrimination exists. He gave a number of recommendations to reduce discrimination.

Back to Top

Case Studies: What Works

Many of the issues described so far are major societal issues, which lead to a diffuse pipeline of minority students into the workforce. Despite the challenges posed by a narrow pipeline, bold leaders have made significant strides in reducing the gender gap in computing, as we will show through a few examples.

At Harvey Mudd College, a liberal arts college of science and engineering, the college went from the average 12% of women in CS to about 40% in five years by taking these actions:19

  • The introductory CS class was changed to be more holistic, and not assume the students already know how to program. The act of designing systems to solve problems that one cares about is a big motivator. A CS class is required for all students in the first semester.
  • First-year women students are encouraged to attend the Grace Hopper Celebration of Women in Computing, a large (about 4,000 attendees) conference, regardless of major; a large percentage of the students choose to attend. This provides role models and breaks the stereotype about what computing is about: hundreds of happy, successful, technical women attend, at all stages of their careers. Even the women who choose not to study computer science say this event has a huge impact on their career choices, and many do decide to study computer science as a result.
  • A research program was offered after the first year for about 10 female students each year for four years. At this time, it is still true that some students (both male and female) have access to research after the first year.

None of these suggestions offer a particularly new insight alone, but in combination they have had a dramatic and lasting effect. Consider these results:

  • In the same period, enrollment by women at Harvey Mudd increased from 33% to 46%.
  • These changes have attracted more computer science majors overall: Harvey Mudd graduated over 70 CS majors in 2014, compared formerly with 25 to 30.
  • In 2014, 56% of those graduating with engineering degrees from Mudd were female. The proportion of graduating CS majors who were female was 39.4%.

Taken together, they break the stereotypes that girls often have about CS, they provide confidence and role models, and they show all students, not just women, the impact they can have in computer science.

Stereotypes make us feel as though we have useful information about people's strengths, weaknesses, and personal characteristics.

The importance of collecting and publishing data is illustrated by the example of the science faculty at MIT. Around 1993, a few of the senior women faculty in science at MIT felt there was bias in the way lab space was allocated, committee appointments were made, and so on. They were able to get support from the dean, and a comprehensive study was undertaken, starting in 1995, which showed that indeed, in some of the science departments, there were differences in salary, resources, awards, and allocated space between men and women with similar accomplishments. The committee then established goals to address these issues. An important factor in the success of the study was the use of data, and the inclusion of both senior women faculty and men who were or had been department chairs, in the committees. A report was published in 1999, and it is all the more remarkable for its openness.31 Importantly, part of the follow-up was to continue to collect and review data. A follow-up study in 2011 showed distinct progress: for example, a woman was hired as president of the institute. Hiring was increased: women represented about 8% of faculty for the 10, or even 20, years prior to the study. Women faculty increased from 30 to 52 in science, and 32 to 60 in engineering; and women now hold some senior administrative positions.

To give an example of the contribution of diversity to team success, the closed captioning feature of You-Tube at Google was developed and its release driven by a deaf engineer, Ken Harrenstien.20 Closed captioning turned out to have a huge business impact, beyond the community of those who cannot hear; well beyond what was anticipated. It happened, in large part, because of the efforts and advocacy of Harrenstien and the accessibility engineer who worked with him, Naomi Black.

Back to Top


Diversity is important to organizations that innovate, but the culture of an organization determines whether minority members of the community can thrive. Computer science as a discipline has not done well in attracting and retaining women and minority practitioners in any proportional scale to population representation: the percentage of BS degrees conferred to women peaked in 1986 and is still on a downturn.34 People from minority groups have contributed at the highest level to the CS discipline since its inception, despite obstacles they have faced, including limited visibility of their achievements. As noted earlier, as little as 1% of bias in ratings can result in reduced promotion rates and a skewed population of the top levels of organizations.29 This makes awareness of the reproducible relevant research extremely important.b

To summarize, things that can be done by leaders to make diverse teams effective:

Make data available. Note the stunning changes that were effected as a result of the MIT Science Faculty Study, because of the insights provided by the data collected, and the open publication of the results.31

Create an atmosphere of trust. Recall that women who stay in engineering are engaged and hopeful.8

Pay attention to critical mass.25,41,44 In order to attain critical mass, organizations should take actions to remove bias from the hiring and promotion processes.

Provide a credible narrative. Provide opportunities for everyone to see themselves as successful: to meet more experienced, successful people, similar to themselves, and who have faced the same barriers.30,41 Know the history of minority contributors to the field, and make sure these achievements are known.

Adopt an expandable view of intelligence: demonstrate you believe that skills can be learned.41

Know your own biases. Read some of the literature about unconscious bias and about the IAT, and then take the Implicit Attitude test5 at https://implicit.harvard.edu/implicit/.

Embrace differences: recall the women in the management study,9 and as a manager, pay attention to differences in needs by individuals.

Foster intergroup conversations as learning opportunities.18

Remember the subject of a bias is not always aware of the effects on him or her.41

For organizations, it is important for individuals in key positions to sponsor promising women and minorities.10,24

An organization that says "we value diversity" is more trusted than one that says "we are color blind."37

Values affirmation has been shown to have long enduring positive effects on classroom performance.12,41

Before important decisions, make sure you are well fed: in one study, subjects were tested for their bias against homosexuals. Half of the subjects drank lemonade with sugar before the test; the other half drank lemonade with a sugar substitute. The subjects who had sugar showed less bias than those who had a sugar substitute.16

Back to Top


I have conducted this research personally because of my interest in the subject, and any omissions are in-advertent. I would like to thank the many people who have sent me relevant literature, but most especially Caroline Simard, who has studied this area extensively (Note that the body of relevant research is larger than what is cited here.) My particular thanks to Robin Jeffries and Brian Welle for their critical reading of and comments on this article, to Bryant York for the reference to the computer scientists of the Black Diaspora, and to Alan Eustace for his instrumental role in making diversity a priority at Google.

Back to Top


For a full reference list, please refer to Supplemental Materials in the ACM Digital Library or to my blog http://rule-of-one.blogspot.com.

1. The Ada Project. Pioneering Women in Computing Technology. Carnegie Mellon University; http://www.women.cs.cmu.edu/ada/Resources/Women/

2. Ambady, N., Shih, M., Kim, A., Pittinsky, T.L. Stereotype susceptibility in children: Effects of identity activation on quantitative performance. Psychological Science 12, 5 (Sept. 2001), 385–390.

3. Aronson, J., Lustina, M.J., Good, C., Keough, K., Steele, C.M. and Brown, J. When white men can't do math: Necessary and sufficient factors in stereotype threat. J. Experimental Social Psychology 35 (1999), 29–46.

4. Ashcraft, C. and Breitzman, A. Who Invents IT? An Analysis of Women's Participation in Information Technology Patenting. NCWIT Report, Dec. 2006.

5. Banaji, M.R. and Greenwald, A.G. Blindspot: The Hidden Biases of Good People. Delacorte Press, 2013.

6. Barres, B. Commentary: Does gender matter? Nature 442 (Aug. 2006), 133–136.

7. Bateson, M., Nettle, D. and Roberts, G. Cues of being watched enhance cooperation in a real-world setting. Biology Letters 2 (June 27, 2006), 412–414.

8. Buse, K.R. Why They Stay: Individual Career Factors Predicting Career Commitment for Women Engineers. Social Science Research Network, May 2011.

9. Carter, N.M. and Silva, C. Pipeline's Broken Promise. Catalyst Report, 2010.

10. Carter, N.M. and Silva, C. Myth of the Ideal Worker. Catalyst Study, Oct. 2011.

11. Center for Work Life Policy. Off-Ramped Women May be the Answer to Japan's Demographic Crisis Finds New Study from the Center for Work-Life Policy, November 2011.

12. Cohen, G.L., Garcia, J., Apfel, N. and Master, A. Reducing the racial achievement gap: A social-psychological intervention. Science 313, 5791 (Sept. 2006), 13071310.

13. Cohen, G.L., Steele, C.M. and Ross, L.D. The mentor's dilemma: Providing critical feedback across the racial divide. Personality and Social Psychology Bulletin 25 (1999), 1302–1318.

14. Croizet, J.C., Deprés, G., Gauzins, M.E., Huguet, P., Leyens, J.P., Méot, A. Stereotype threat undermines intellectual performance by triggering a disruptive mental load. J. Personal and Social Psychology 30, 6 (June 2004), 721–731.

15. Desvaux, G., Devillard-Hoellinger, S. and SancierSultan, S. Women Matter: Women at the top of corporations: Making it happen. McKinsey Report, 2010.

16. Gailliot, M.T., Peruche, B.M., Plant, E.A., and Baumeister, R.F. Stereotypes and prejudice in the blood: Sucrose drinks reduce prejudice and stereotyping. J. Experimental Psychology, Jan. 2009.

17. Gladwell, M. Blink. Little Brown, 2005.

18. Goff, P.A., Steele, C.M. and Davies, P.G. The space between us: Stereotype threat and distance in interracial contexts. J. Personality and Social Psychology 94, 1 (2008), 91–107.

19. Haffner, K. Giving women the access code. New York Times (Apr. 2, 2012); http://www.nytimes.com/2012/04/03/science/givingwomentheaccess-code.html

20. Harrenstien, K. Finally, Caption Playback. Google Video Blog (Sept. 2006); http://googlevideo.blogspot.com/2006/09/finallycaptionplayback.html

21. Heilman, M.E. and Chen, J.J. Same behavior, different consequences: Reactions to men's and women's altruistic citizenship behavior. J. Applied Psychology 90, 3 (2005), 431–441.

22. Heilman, M.E., Chen, J.J. No credit where credit Is due: Attributional rationalization of women's success in male-female teams. J. Applied Psychology 90, 5 (2005), 905–916.

23. Herring, C. Does diversity pay? Race, gender, and the business case for diversity. American Sociological Review, Apr. 2009.

24. Hewlett, S.A. Leader-Chivée, L., Sumberg, K., Fredman, C. and Ho, C. Sponsor Effect. Center for Talent Innovation, U.K., 2012.

25. Inzlicht, M. and Ben-Zeev, T. A threatening intellectual environment: Why females are susceptible to experiencing problem-solving deficits in the presence of males. Psychological Science 11 (2000), 365–371.

26. Joy, L., Carter, N.M., Wagner, H.M., and Narayanan, S. The Bottom Line: Corporate Performance and Women's Representation on Boards. Catalyst Report, Oct. 2007.

27. Judge, T.A. and Cable, D.M. When it comes to pay, do the thin win? The effect of weight on pay for men and women. Journal of Applied Psychology 96, 1 (Jan. 2011), 95–112.

28. Kleiman, K. Eniac Programmers Project; http://eniacprogrammers.org/

29. Martell, R.F., Lane, D.M., Emrich, C. Male-female differences: A computer simulation. American Psychologist 51, 2 (Feb. 1996), 157–158.

30. McIntyre, R.B., Paulson, R.M. and Lord, C.G. Alleviating women's mathematics stereotype threat through salience of group achievements. J. Experimental Social Psychology 39 (2003), 83–90.

31. MIT Faculty Newsletter: A study on the status of women faculty in science at MIT, March 1999; http://web.mit.edu/fnl/women/women.html

32. Moss-Racusin, C.A., Dovidio, J.F., Brescoll, V.L., Graham, M.J. and Handelsman, J. Science faculty's subtle gender biases favor male students. In Proceedings of the National Academy of Science (Aug 2012).

33. NASSCOM-Mercer. Gender Inclusivity in India: Building Empowered Organisations, 2009.

34. Mitchell, R.L. Women computer science grads: The bump before the decline, Computerworld Blogs, (April 2013); http://blogs.computerworld.com/itcareers/21993/womencomputersciencevisualtrendline

35. Pieper, J. Leisure, the Basis of Culture. Ignatius Press, 2009.

36. Pittinsky, T.L., Shih, M., Ambady, N. Identity adaptiveness: Affect across multiple identities. J. Social Issues 55 (1999), 503–518.

37. Purdie-Vaughns, V., Steele, C.M., Davies, P.G., Ditlmann, R. and Crosby, J.R. Social identity contingencies: How diversity cues signal threat or safety for African Americans in mainstream institutions. J. Personality and Social Psychology 94, 4 (2008), 615–630.

38. Schilt, K. and Wiswall, M. Before and after: Gender transitions, human capital, and workplace experiences. The B.E. J. Economic Analysis and Policy 8, 1 (2008), 1–28.

39. Spencer, S.J., Steele, C.M., Quin, D.M. Stereotype threat and women's math performance. J. Experimental Social Psychology 35 (1999), 4–28.

40. Steele, C.M. and Aronson, J. Stereotype threat and the intellectual test performance of African Americans. J. Personality and Psychology 69, 5 (Nov. 1995), 797–811.

41. Steele, C. Whistling Vivaldi. W. W. Norton, 2011.

42. Stone, J., Lynch, C.I., Sjomeling, M., Darley, J.M. Stereotype threat effects on black and white athletic performance. J. Personality and Social Psychology 77, 6 (1999), 1213–1227.

43. Tannen, D. The power of talk: Who gets heard and why. Harvard Business Review, Sept. 1995.

44. Totenberg, N. Sandra Day O'Connor's Supreme Legacy: First Female High Court Justice Reflects on 22 Years on Bench. All Things Considered (May 14, 2003); http://www.npr.org/templates/story/story.php?storyId=1261400

45. Treisman, U. Studying students studying calculus: A look at the lives of minority mathematics students in college. The College Mathematics Journal 23 (1992), 362–372.

46. Vedantam, S. The Hidden Brain. Random House Publishing Group, 2010.

47. Williams, S. Computer Scientists of the African Diaspora. SUNY Buffalo; http://www.math.buffalo.edu/mad/computer-science/cs-peeps.html

48. Woolley, A.W. Chabris, C.F., Pentland, A., Hashmi, N. and Malone, T.W. Evidence for a collective intelligence factor in the performance of human groups. Science Magazine, 330 (Oct. 29, 2010), 686–688; DOI: 10.1126/science.1193147.

49. Woolley, A. and Malone, T. Defend your research: What makes a team smarter? More women. Harvard Business Review Report, June 2011.

50. Zweben, S. and Bizot, B. 2012 Taulbee Survey: Strong increases in undergraduate CS enrollment and degree production; record degree production at doctoral level. Computing Research News 25, 5 (May 2013).

Back to Top


Beryl Nelson (beryl.nelson@gmail.com) is a software engineering manager at Google, Mountain View, CA.

Back to Top


a. Many studies referenced here refer to women, but these results should largely be considered to apply across all axes of diversity; for example, gender, cultural and national origin, sexual orientation, age, educational background, religion, and other life experiences. It is more difficult to study differences that are not externally visible, such as differences in economic class, than visible differences like gender or race, but these less-visible differences are also important to consider in conversations about diversity.

b. This research is particularly meaningful to me because of my experience of living in foreign countries from 1995–2014. I have come to understand that people see me through the lens of their own experiences and cultural attitudes. Similarly, I have come to better understand the biases that my own native cutlure has toward these other cultures, sometimes causing fundamental misunderstandings. Probably the most effective training that a person can have in terms of understanding what it is to be a minority is to become one: live in a place where you are the only one like you. However, these studies can help to understand some of what it means in day to day life to be Other.

Back to Top


F1Figure 1. Women at the top of corporations: Making it happen. Courtesy of McKinsey Report.15

F2Figure 2. Financial performance at companies with three or more women board directors (WBD).

F3Figure 3. Collective Intelligence vs. Team Composition.

F4Figure 4. Coffee station pictures and money collected. Courtesy of Bateson et al.7

F5Figure 5. Reasons for leaving a first job. Courtesy of Carter et al.9

Back to Top

Copyright held by author.

The Digital Library is published by the Association for Computing Machinery. Copyright © 2014 ACM, Inc.


Chunhua Liao

I am stunned by the publishing of this article by a prestigious academic magazine from ACM. This article first defines diversity as "heterogeneous in meaningful ways, for example, in skill set, education, work experiences, perspectives on a problem, cultural orientation, and so forth". The author did not (dare to) list genders and races in his/her definition of meaningful heterogeneous qualities. But for the rest of the article, all he/she was talking about is data about diversity defined by genders and skin colors, such as "19.2% of Ph.D.'s were awarded to female candidates" and "5.3% of BS degrees were awarded to African American candidates". I cannot find a single data source studying diverse skills, education, work experiences and so on. I cannot help but wondering: is the author assuming people with the same gender will automatically not diverse? Is the author also assuming people in the same skin color can not be possibly diverse in their qualities?

If the author really believes in his/her definition about diversity in qualities of people and wants to fight stereotypes based on genders and colors, the presented data would be totally different and gender/color blind. For example, in the case of testing how innovative a team is, the meaningful heterogeneous qualities would be the different perspectives for a domain, the degree of risk averse in the team members, the thinking patterns of top-down and bottom-up, the preference between incremental and radical changes, and so on. All these qualities are far more meaningful than the superficial, stereotyped gender and race differences in the team members.

The author seems lack the basic understanding of (or knowingfully ignore) why stereotypes happens at all. I would point to the concept of associative memory used by human brains. When two concepts often come to a human brain, the connection between these two concepts will be enforced. Mentioning one concept will automatically trigger the onset of the other concept in our minds. Essentially, neurons (brain cells) send signals to each others. The more signals sent between two neurons, the stronger the connection grows. Our brains re-wires physical structures based on the input. For example, if we often see a suspect wearing a red hat committing burglaries in the news. Our brains will automatically associate red hat with burglaries and remind us to be cautious when seeing a person wearing a red hat. These kind of associations by themselves are just our intuitions, which are not necessarily good or bad. Definitely we should not feel ashamed by these intuitions at all. What matters is how we use our logic and reasoning to make decisions from intuitions. Rather than weakening the stereotyped associations based on genders or races, what the author suggest (using genders and colors to measure and increase diversity) would in fact further strengthen the exact associations the author claims to fight against. In the end, the author is promoting gender and race based discrimination.

I don't know this article could pass the peer review. I am very disappointed by the publishing of this article and also disappointed by ACM.

Chunhua Liao

One more point:

Maybe the biggest flaw in the article is something the author already admits: all cited studies about the benefits of gender/race-based diversity were correctional. None of them were designed to prove causal relationship, which I doubt they could ever be unless we use meaningful qualifies to re-design the studies. Still, while the author says the results only "to a degree that demands attention", the actions promoted are alarming to me: acting now using gender/races to improve diversity even when there is no controlled, conclusive studies showing casual relation between gender/races and job outcomes.

I think social policies are like medical drugs. They must pass rigid controlled studies before fully pushed for implementation. Having correctional studies are far from rushing to adapt the policies (drugs). The consequences can be very severe.

Beryl Nelson

As I read it, the fundamental basis for all of Dr. Liaos comments comes from his strong opposition to any actions that would promote diversity based on gender or skin color. I will try to address this and several other of the issues that he mentions.

It is very common for people to believe that they can extrapolate from their own personal experiences; we see this in asking minority leaders to explain how they succeeded, for example. I also have much personal experience, having lived 19 years in total overseas; so that I have lived in 5 countries on 3 continents and worked in 4. I have raised 3 accomplished daughters whose coloring is not like mine. However, most of us (and I include myself) do not have enough personal experiences to be statistically significant. Moreover, usually it is impossible to experience life as two different demographic groups; e.g. black and white (the exception being blacks with light skin who make a decision to pass as white; this has been written about extensively); or as male and female (except for transgenders).

The primary purpose of this article is to assemble in one place the state of the art of quantitative repeatable research in a number of subfields of diversity research. I maintain that, although personal experience and anecdotes can give us some curiosity about what is happening at a larger scale, studies at scale can help us to understand the effects more fundamentally in society.

It is true that the benefits of diversity so far have been mostly correlational studies (I believe that Dr. Liao meant correlational, not correctional, in his comment). This is because it is extremely difficult to construct an experiment showing benefits at a scale which would show proof in the real world. There is not a statistically significant number of large organizations run by diverse leadership, on any of the visible metrics, such as gender, country of origin, race.

Anecdotally, there have been interesting experiments. For example, when I lived in Japan many years ago, there was a very popular indoor swimming and artificial wave resort called Deep Blue, in Yokohama. The company which built this was in the business of steel, and when the demand for steel for shipbuilding reduced, the companys leaders took the unusual step of calling a brainstorming meeting, to discuss what else could they do with their expertise, and they included everyone in the company, from top to bottom. This included secretaries and people who worked in deliveries. The result was this very popular year-round water resort. This was considered a success in innovative thinking at the time. Another example is a company which tried a business model in which offices could not exceed a certain size; perhaps it was around 150 people. Once offices exceeded that size, a new office would be created and split off. Importantly, every office had to include every kind of function. However, these are isolated organizational experiments, and it is not really possible to extrapolate from small numbers of instances like these.

However, since the publication of my review article, there has been a new research article published by Wooley and Mallone (authors of cited research), which show the characteristics of highly effective teams: http://www.nytimes.com/2015/01/18/opinion/sunday/why-some-teams-are-smarter-than-others.html
They are making significant progress at defining the traits of successful innovative teams, which will go a long way to understanding how diverse teams actually make use of their differences, when successful. Their findings: the most successful teams included members who communicated a lot, participated equally, and possessed good emotion-reading skills. This is certainly an encouraging result.

I agree with Dr. Liao that it would be extremely interesting to have more studies of diversity along traits that one cannot see in a person. However, there is little literature about this, just because it is so hard to define in a stable way what a trait means if you cannot see it. If you call someone risk averse, does that vary by the situation? If one person is willing to do free rock climbing, but is very traditional about educational values; and another person is not willing to take such physical risks, but very open to a son or daughter taking a non-traditional path, not going to university: which of these is risk averse, or risk friendly? It would be hard to create a reliable metric. There are, however, studies which do associate risk taking with gender. Such associations are not absolute, but the correlations are strong. Because of such difficulties, most of the literature describing bias and stereotype threat uses traits that are visible or known, such gender or race or sexual orientation, or in some cases socio-economic status (by using addresses in different neighbourhoods, for example). It would indeed be interesting to see other traits represented in the literature.

About the attention to gender and race, it has been shown quite clearly that the human capital from women and from blacks is often ignored. The MIT Study mentioned in my article, for example, showed that, until there was data showing such a trend, the university was routinely offering less support to accomplished senior women than to similarly accomplished men. This shows quite clearly that an organization cannot understand how it is doing in valuing human capital unless it has data about it.

Being Asian is actually a positive for stereotypes relating to math and science. But being black has many downsides in the US. This is one of many articles which shows ways in which being black affects almost every part of life: health care outcomes, work opportunities. This is a scary effect; imagine not being offered needed medical care because of the way you look. http://www.nytimes.com/2015/01/04/upshot/the-measuring-sticks-of-racial-bias-.html

Dr. Liao also asserts that addressing gender and race discrimination would create more discrimination. This can indeed happen, and needs to be addressed. When people within an organization believe that someone received special treatment on hiring, that can be of detriment to the minority hire: it means that he or she needs to prove him or herself every day on the job in a way that the majority hires do not need to. An organization must be clear in stating that everyone is held to the same standards in order to be successful in building an effective workforce.

Talking about bias and stereotypes can also have a negative effect, but this can be mitigated. See for example this recent article: http://www.nytimes.com/2014/12/07/opinion/sunday/adam-grant-and-sheryl-sandberg-on-discrimination-at-work.html
By adding the information, Most people work hard to counter their stereotypes, the subjects of the experiments were led to also try to mitigate stereotypes.

The work of Harvey Mudd College shows that understanding the needs of women students has made their Computer Science Department more effective overall: enrollments are up in total, not just in the proportion of female students.

I do not expect this article to change the opinion of every reader, but rather to make the original research more available to interested people. It has been shown that being presented with scientific evidence about diversity has almost no effect on most peoples personal beliefs. This article, for example, reports on a study showing that men most often reject scientific evidence of gender bias: http://scienceblog.com/76268/men-women-respond-gender-bias-stem/#yCuFI5Yw3ei8A7Zc.97
Anecdotally, even people who were clearly affected by bias sometimes deny the effect. For example, Rear Admiral Grace Hopper, who was a pioneer in early computing and led a team at Harvard during World War II, was evicted from both the Navy and Harvard after the war, when womens contributions were no longer wanted. It was a couple years before Mauchly and Eckert invited her to join their team, building what later became the UNIVAC; she led the software development team, and was the first person to talk about and build a compiler. Many years later she rejoined the Navy. When she was asked later in life what she thought about womens lib, she said, I dont know that much about it because I didnt have to worry about it; I was in the Navy. This despite the Navy rejecting her in 1945. The fact that she was a respected and accomplished Computer Scientist does not mean that she was correct about there not having been bias. You can see her comments to that effect in this recently produced documentary: http://fivethirtyeight.com/features/the-queen-of-code/

A very disturbing finding by social scientists is represented in this recent news article
If you believe that the world is fair, and you are in a situation in which something bad is happening to someone, and you cannot do anything to change it, you blame the victim. The original experiment involved giving a shock to a woman if she gave a wrong answer. If the observer had the ability to have the shocks stop, the observer sympathized. If the observer did not have the ability to stop the shocks, then the observer started to blame the woman. This is very disturbing, because we all do want to create a world that is fair. We must guard against this effect if we are to make the world truly fair.

Displaying all 3 comments

Read CACM in a mobile app!
ACM Logo
  • ACM CACM apps for iPad, iPhone, and Android
  • ACM Digital Library app for iOS, Android, and Windows
  • Download free and sign in with ACM Web Account
Find the app for your mobile device