Lessons for Investigators from Studies of Judicial Decisions

I sometimes train company managers how to investigate claims of discrimination or harassment.  The most thoughtful students, especially if they’ve had some prior experience conducting investigations, eventually present this concern:  What if both sides seem to be telling the truth?  How can I know which way I should decide?  How can I make sure I remain objective and impartial?  And (most quietly) how can I make sure my investigation doesn’t reflect some bias of my own?  For this article, I’ll discuss only one facet of this enormous topic:  what two statistical studies of judicial rulings suggest about the role of race and gender in the adjudication of discrimination and harassment claims.  The studies in question are Pat K. Chew and Robert E. Kelley, Myth of the Color-Blind Judge:  An Empirical Analysis of Racial Harassment Cases (Washington University Law Review, Vol. 86, p. 1117 (2009)), and Jennifer L. Peresie, Female Judges Matter:  Gender and Collegial Decisionmaking in the Federal Appellate Courts (The Yale Law Journal, Vol. 114, p. 1759 (2005)).

Chew and Kelley examined the role of federal judges’ race and gender in motions for summary judgment in 400 employment-based racial harassment claims decided between 1981 and 2003, where the question before the judge was whether the complaining employee had presented enough evidence to be entitled to trial before a jury.  They assert that African American judges in these cases were significantly more likely to rule in the employee’s favor on such motions, regardless of the employee’s race, than were Caucasian or Hispanic judges – they denied the motions, effectively authorizing racial harassment claims to go to trial in 45% of the racial harassment cases before them, compared to rates of 20.6% for Caucasian judges and 19% for Hispanic judges.  Chew and Kelley also reported, among other findings, that allegations of racial slurs or that supervisors as well as coworkers were engaging in harassment significantly increased the probability of a ruling in the employee’s favor, suggesting close judicial attention to the details of each case.  On the other hand, neither the gender of the presiding judge nor the race of the complaining employee could be said, based on the data under review, to have a significant effect on the outcome of the motions in these racial harassment claims.  It does not appear from the authors’ analysis that certain legal developments in the field during the period of the study, including the introduction of the Faragher-Ellerth affirmative defense in 1998, were considered as part of the analysis.

Peresie, by contrast, reviewed 556 employment cases involving allegations of sexual harassment or sex discrimination between 1999 and 2001 to assess the effect of having at least one female judge on the assigned panel of three federal judges reviewing the case.   Restating some of Peresie’s data for direct comparison to Chew and Kelley’s report, it appears that federal appeals panels with at least one female judge ruled in favor of the sexual harassment plaintiff (whether male or female) 35.5% of the time (87 out of 245 cases), while all-male appellate panels ruled in the plaintiff’s favor 17% of the time (53 out of 311 cases).  Peresie also observed that, based on this data, neither the race of judges, nor the gender of the plaintiff, could be said to have a significant effect on the rulings in these sexual harassment claims.  However, allegations of sexualized behavior – although not a legal requirement in such claims – significantly increased the likelihood of a ruling in the employee’s favor.

Both studies note that they see no evidence in these samples of judges showing favoritism for or against members of any race or gender, and instead find statistically significant evidence that the outcomes depend on the facts present in any particular case.  They also suggest no appreciable difference in how African American and Caucasian judges approach sexual harassment cases, or in how male and female judges approach racial harassment cases.   These studies are also rightly cautious not to extrapolate beyond the narrow set of employment discrimination lawsuits they set to examine.  However, Chew, Kelley, and Peresie do seem to argue that where there is room for discretion, such as in deciding that there was enough evidence for a jury to conclude that conduct was sufficiently severe or pervasive to alter the employee’s terms and conditions of employment, African American judges exercise that discretion in racial harassment claims, and female judges do so in sexual harassment claims, in manners that differ subtly from their colleagues.  Because the race or gender of the plaintiffs in these cases does not affect the outcome, this does not appear to be evidence of bias, but rather the lack of it.  Instead, one plausible interpretation of these results is that, on average, African American judges may have a broader perspective on what can qualify as racial harassment in general, and female judges may have a broader perspective on what can qualify as sexual harassment in general – a difference in legal interpretation – than certain of their colleagues.

I have concerns about the authors’ conjectures and conclusions in both studies that we need not dwell on here.  Suffice it to say that given the data and computational tools at their disposal, neither study was really equipped to identify the reasons for these subtle differences in rulings, which inevitably will depend on the unique facts of each case.   In addition, since cases are assigned to judges randomly, conjectures about judicial psychology are unlikely to be much help to managers, executives, or human resources professionals.  Of more interest, at least for investigators, may be the juxtaposition of the judges’ evident impartiality with their careful attention to the facts.

Psychologist James Reason observed in Human Error, his 1990 examination of the psychological and organizational factors contributing to major accidents, that “the more entrenched the hypothesis, the more selective becomes the working definition of what is relevant” (p. 169).  People who are already committed to a certain outcome tend to ignore or dismiss facts that stand in their way; those less certain about how things should come out attend more to the context of the dispute.  In my experience, this fine-grained attention is common in judicial rulings, and it isn’t unusual for decisions like those studied by Chew, Kelley, and Peresie to mention and consider factual details that the attorneys didn’t highlight in their papers, but were otherwise evident from the record.

There are a number of reasons why such an approach is helpful to ensuring a fair outcome.  The parties to a dispute (as well as their attorneys) become increasingly entrenched in their positions as they are called upon to articulate and defend their views – we get convinced by our own arguments over time, partly because we hear ourselves making them, and partly because we spend so much more time with our own arguments than our adversaries’ as we ready for trial.  A neutral arbiter therefore needs to recognize that both sides, in their increasing partisanship, may overlook certain facts and considerations relevant to the overall dispute despite their intentions of candor.

Further, although the parties to a dispute become emotionally invested over time, and although this investment may provoke a certain amount of unconscious revisionism, few people have the opportunity, ingenuity, or (potentially criminal) intention to review and sanitize not only all the documents in the case, but also their recollection of facts, to be entirely consistent with their current outlook.  When all sides have argued themselves to exhaustion, minute but telling details still tend to be buried in the record of the case, undistorted by the parties’ sometimes heated rhetoric.

Third, it is vitally important for any investigator to appear to be fair to all sides.  Rendering a decision that relies solely on what the disputants have said invariably exposes the investigator to accusations of favoritism or bias from the loser; a decision that relies at least partially on facts that neither side mentioned is harder to attack in this way.

There is, of course, more that an investigator can undertake in the interest of integrity.  One technique is to examine one’s own definition of relevance in a particular investigation, as well as that employed by either side.  Does one side have any especially broad, narrow, or lopsided view of relevance?  Is there any overlap between the parties?  Are there any questions that ordinarily get asked in these situations, but haven’t been asked here?  Does one side simply refuse to hear the other side’s perspective?  There will be some situations where one side’s view is so narrow as to be either irrational or transparently political – as in the case before Solomon – and reveal the appropriate outcome.  And there will be other situations where examining the investigator’s own notion of relevance will reveal previously overlooked details.  In particular, if someone signals that they don’t feel they’re being heard, and that their concerns are being ignored or dismissed out of hand, there may be good reason to ask follow-up questions to better understand why they believe those ignored details are so important; such conversations will clarify, if nothing else, the background and perspective of one of the parties.  For my part, I maintain a written checklist of questions (numbering more than 40 at this point, though not all will apply to a given case) that I sometimes consult when reviewing delicate employee termination decisions with a client.   When not testing the soundness of a termination decision, the checklist doubles as a fair training tool.

This is not to suggest that the judges studied by Chew, Kelley, and Peresie consciously adopted such approaches in the cases before them.  But since these studies were undertaken at least in part to look for correlations between judges’ race and sex and their decisions in discrimination cases, the fact that neither study found any evidence of bias, and only, arguably, subtle differences in legal interpretation, is quite noteworthy.  It suggests that fairness is, with practice, an achievable goal, and that careful attention to detail may be one strategy to help achieve it.