Alternative Exploratory Research Designs in Litigation
12. Copyright 2014 Litigation Psychology, LLC. All Rights Reserved.
6. 6 Hispanic females, and then again for Hispanic males – the number of test respondents required becomes even larger. Without delving into the complex topic of sampling techniques, it suffices to point out that, in this type of research, generally a sample size of test respondents exceeding one hundred is required for the determination of useful juror profiles. In practice, research projects of this type follow the steps outlined below: 1. First, select a group of about 120-140 respondents comprising representative cross-sections of jury- eligible persons from the venue (i.e., ensuring a representative cross section of ethnic, marital, age, educational and other similar categories); 2. Prepare videotaped presentations for each important party that capture the essential claims and responses in the case; these presentations should normally be about 45-60 minutes each, based on the key issues and evidence involved in the litigation; 3. Schedule multiple Focus Group sessions of about 4-5 hours each, with each one using 20- 30 test respondents; thus, using a sample size of 120, four Focus Groups of 30 subjects each or five groups of 24 subjects each would be optimal (while it would in principle be possible to use three Focus Groups of 40 subjects each, it should be remembered that, in order to conduct focus sessions with group discussion, the number of respondents in each group should not be too large); 4. Develop a comprehensive pre-test that includes measurements of any and all relevant pre-existing characteristics (demographics, experiences, beliefs, attitudes and personality variables) that will or could be considered in actual jury selection. Thus, for a toxic case, one might inquire as to hospitalizations, illnesses, past toxic exposure, diet and health behaviors, etc. This pre-test is a comprehensive and labor- intensive amalgamation of measurements that could conceivably be instrumental in differentiating plaintiff and defendant (or low- damage versus high-damage) jurors; 5. Develop a post-test administered after respondents have seen the presentation tapes that assesses verdict and damages dispositions (along with any other case-related queries that may be useful); 6. Implementation, consisting of (for each of the multiple 30-40 member Focus Groups used) a) administration of the pre-test in (4) above; b) subjecting test respondents to the presentations in (2); c) administration of the post-test in (5); and d) focus sessions in which test respondents provide their opinions of the case. 7. Data Analysis: Using various statistical correlational techniques, ascertain the existence of significant relationships between the pre-test variables measured in (4) and the verdict and damages outcomes measured in (5) to derive juror profiles. In this research, the availability of “Focus Group” data – normally, the comments offered by respondents describing their reactions to the case themes and issues – is essentially a by-product of the process of obtaining the juror profiles. That is to say, the real “meat” of the project is the considerable work involved in the steps above leading to the profiles ((4), (5), (6 a-c) and (7)), while the use of focus sessions (6 (d)) is an added “bonus.”
11. 11 About the Author Dr. Speckart received his Ph.D. in Psychology from UCLA in 1984 with a specialization in personality measurement. He has been active in the jury consulting field since 1983, and has conducted over 600 mock trials and focus groups in pre-trial research for numerous types of litigation. Dr. Speckart has worked with litigators in over 150 jury selections, beginning with Dalkon Shields cases in 1983, the Agent Orange litigation in 1984, and Exxon Valdez litigation in 1994. His area of emphasis has shifted to patent litigation over the past decade as a result of increased demand for assistance in this complex area of jury psychology.
10. 10 intentions of the researchers in the former entail damages. In addition, rarely (if ever) do we see damages in projects run by plaintiffs that are not significantly higher than those awarded in trial. Even in Exploratory Research one cannot escape the ultimate requirement that the research is intended to predict. While such research is not designed to forecast verdict and damages outcomes (the typical requirement for “prediction”) it is still nonetheless predicting the subjective responses of jurors that will presumably occur in the real trial – otherwise, what utility could the research possibly provide to the litigator? And, if prediction of any kind is in fact required, then we are operating at the highest level of scientific inquiry, and suitable qualifications for conducting the research should be ensured.
1. George Speckart, Ph.D. “We are drowning in information but starving for wisdom.” E. O. Wilson Alternative Exploratory Research Designs in Litigation
3. 3 presentations; c) post-test measurements to gauge their reactions to the case; and d) the focus session itself, in which respondents are queried as to their reactions to the case, and the more peripheral beliefs that regulate such reactions. Thus, the design may be loosely conceived in terms of a “recruit/pre-test/presentation/post-test/focus session” structure, which represents perhaps the most rudimentary of jury research approaches, and indeed these types of projects are often carried out informally, with little (or no) scientific rigor. The utility of this type of research arises from its capacity to determine what is important to respondents in making a verdict decision, versus that which is hypothesized to be important to jurors but in reality is not. This “separating the wheat from the chaff” or differentiating “correct” from “clever” may play a crucial role in theme development, since some case issues, themes and arguments may seem to be important on an a priori basis but ultimately do not in fact have any impact on the actual verdict and damages decisions made by jurors. Moreover, jurors frequently interject issues that are important to them but that were unanticipated by the trial team, lending these projects a kind of serendipity that can beneficially “feed” trial strategy. Cost effectiveness of these exercises can be considerable, as themes that were hypothesized to be important but found to be ineffective later become discarded, reducing the “load” on the trial team and concomitant discovery costs. As one attorney put it, “I ran a focus group and found out I did not need a $75,000 expert that I was expecting to use.” Streamlining the thematic structure of a case through Focus Group research may therefore have a joint effect of making the preparation both more economical and more effective by reducing the themes to only those with maximal traction in terms of persuasive impact. The degree of predictive validity – how well the research can forecast actual juror dispositions and reactions in trial – is a complex topic, but generally speaking, the more trial-like the research conditions, the better (more accurate) the prediction. As we shall see in examining the research options that follow, some research designs are more “trial like” than others. For the present purposes, we note that achieving “trial like” conditions will first involve the completeness of the presentations, and the factual/ evidentiary content included therein. Each of the following types of Exploratory Research includes some form of presentation to ground jurors in the facts of the case, and it is the content of these presentations that figures as the most important “...jurors frequently interject issues that are important to them but that were unanticipated by the trial team, lending these projects a kind of serendipity that can beneficially ‘feed’ trial strategy.”
8. 8 track record of impressive performances). The area of person perception – ascertaining the degree of attractiveness and likeability characterizing a given person – is exceptionally vulnerable to bias, and trial teams seem to routinely over-estimate the positive characteristics of their own witnesses and under- estimate those of their opponents. The fallibility increases exponentially when one considers the stressors induced by the courtroom environment, and how such stressors may unexpectedly affect ultimate performance on the courtroom floor. Witness Effectiveness Training with Jurors ameliorates these shortcomings in the following manner: Overall, the intent of the exercise is to recreate a slice of the courtroom proceedings by using a jury in conjunction with attorney advocates who provide mock direct and cross examinations, with breaks between testimony sessions in which the panel is queried as to their reactions to the witness performance on relevant dimensions (nonverbal behavior, mannerisms, body language, tone, speech rate, eye contact, facial expressions, and of course the substance of the testimony as well). These juror reactions, elicited in multiple, successive Focus Group settings (or in a single Focus Group session at the conclusion) are “fed back” to the witness and trial team members in real time via closed circuit television. The implications, lessons, and general information provided by jurors are then used as a basis for final preparation of the witness in additional “sit down” sessions. In other words, in this type of witness training, we let the jurors guide the ultimate witness training, based on what they need to see and hear in order to be fully convinced. These exercises utilize very little (if any) questionnaire measurements, although response scales may be used to “score” witnesses on credibility dimensions (e.g., candor, trustworthiness, likeability, communicativeness, expertise, etc.). Steps are as follows: 1. First, recruit a suitable sample of jury-eligible test respondents who are representative of the trial venire (in this case 18-24 mock jurors are typically sufficient); 2. Develop a) introductory or “basic” presentations for each side that provide essential information about the case as background and b) scripts for mock direct and cross examination for the witness(es) who are participating; 3. Utilize a courtroom-like set up consisting of a judge’s bench, jury box, witness box and counsel table, along with electronic presentation systems to create a realistic trial-like environment; 4. Implementation, consisting of a) having attorneys give respondents the “basic presentations;” b) providing mock direct (or cross, for witnesses who are going to be called adverse); c) excuse the trial team from the “courtroom” and conduct a focus session revealing the witness’s characteristics, strengths, weaknesses, etc. with real time transmission through closed circuit television to a secure room where the trial team and witness view the proceedings with the jurors; d) have the trial team re-enter the “courtroom” and resume with the next set of questions to the witness (if direct was first, then this would be the cross, and vice-versa); e) repeat (c) above; and f) continue repetition of the process as needed for as many witnesses as required. 5. [Optional] Questionnaires may be developed
4. 4 determinant of overall predictive validity. While we will resume consideration of this topic at the conclusion of this paper, it is also noted that research methodology plays a key role in the reliability of the results for purposes of making inferences as to how actual jurors will perceive the case. Special Designs – Graphics Research Especially in cases that are fact-intensive, it becomes strategically imperative to formulate graphics with optimal educational and persuasive content. The critical role of well-designed graphics in maximally effective trial strategy has generally become self-evident, although some litigators cling to the belief that hand-drawn charts created on the fly “communicate better” (they do not). In some instances, however, the need to identify precisely how to best present complex data becomes so involved or ambiguous as to necessitate an Exploratory Research design that exclusively investigates demonstrative exhibits and their overall impact. For example, in the Exxon Valdez litigation, the marine biology issues arising in conjunction with the claims by fisheries, seafood processors, and Native Alaskans generated the production of over 500 graphics by the defense team to communicate all of the issues related to fish and bird populations, water purity, and dissipation of crude oil, to name a few. With nearly a dozen experts having over 40-50 charts and exhibits each, the need to cut down the number of exhibits at trial became a key impediment to forward progress in trial readiness. In a classic case of information reduction, Graphics Research was undertaken to reduce the number of charts and simultaneously elicit the feedback needed from test respondents to whittle down the stockpile of exhibits to under 100. Added to the need for information reduction, however, was the additional requirement that the trial team “meet the jury where they are” in terms of comprehension and assimilation of technical data. In short, there was a tangible need to have fewer and more persuasive graphics. The design parameters used in this research, and recommended generally, are as follows: 1. First, recruit a suitable sample of jury-eligible test respondents (30-50) who are representative of the trial venire; 2. Develop a) introductory or “basic” presentations for each side that provide essential information about the case as background; and b) short (2-3 sentence) vignettes to explain the significance of each graphic or demonstrative exhibit; 3. Implementation, consisting of a) giving to respondents the “basic presentations” and subsequently, b) graphic by graphic, providing the explanatory vignettes in conjunction with each chart or exhibit while simultaneously administering c) a measurement device (questionnaire) that gauges their reactions to each chart or exhibit, both in terms of i) clarity and ii) persuasive impact (other parameters may be measured as well, such as “educational value,” “appropriateness” and so on); and d) a short focus session with each graphic (or in clusters of graphics to save time in situations where there are many hundreds of graphics to
5. 5 assess), in which respondents are queried as to their reactions to the individual charts and exhibits. Data from the these “mini focus sessions” in d) for each exhibit (or cluster) provide the qualitative insight as to how to best modify, refine or re-design each graphic to maximize impact, clarity and overall “juror friendliness” of the exhibit – that is, meeting the jury where they are. Charts and exhibits developed by experts can often be complex and lacking in “jury friendliness.” By contrast, using Focus Group queries developed to ascertain what jurors need to learn and understand, results obtained from each chart or exhibit can be used to enhance communicativeness, facilitate recall, and maximize overall persuasive content as the charts or exhibits are modified based on the findings from the project. Moreover, the quantitative data from the measurement devices (questionnaires) can be used to rank-order exhibits in terms of their overall lucidity, persuasiveness or according to whichever parameters have been implemented in the measurement instruments. Such rank-ordering generally reveals where the best charts are, what styles and approaches to presenting data are most effective, and what not to do by looking at the bottoms of the lists. Special Designs – Juror Profile Research Often a case “is what it is” – there are no substantial means of “spinning” the case fact scenario -- and the real challenge is in picking the right jury. The issue of jury selection entails sophisticated scientific inference, as one is necessarily involved in the prediction of behavior: Will this juror vote one way or the other? How strongly will this person advocate a position in deliberations? The need for “sophisticated scientific inference” in turn requires a consideration of a methodological approach to carrying out valid research to accomplish the objectives of prediction. It is worth mentioning that those who decide to use this type of research are perhaps the most sophisticated of litigation teams who recognize that effective jury selection is much more than “I like this person” or “I don’t like that person.” Prediction of behavior is, in fact, the highest level of scientific achievement, posing substantial challenges even for the most experienced of research psychologists. Accordingly, specific research design objectives must be met in order to obtain information that can be relied upon in the courtroom. The use of “Focus Group” research in this instance, as Exploratory Research, may be a bit misleading, but the nature of the methodology ends up resembling Focus Group research in the final analysis. The first methodological issue that requires consideration is the need for large samples of test respondents. In order to scientifically delineate valid juror profiles, one first needs to be able to identify the degree of risk associated with various categories. For example, if one needs to be able to assign a degree of risk for, let’s say, Hispanic females versus Hispanic males, it is necessary to have a sample of respondents from each sub-group to determine how that sub-group is likely to vote. When more precision is needed – for example, differentiating married from divorced
2. 2 2 P re-trial or “jury research” is sometimes bifurcated into two categories, the first being Exploratory Research – projects designed to initially “see what’s out there,” investigate the substantive terrain of a case, and find out “what sticks” thematically in terms of case arguments, issues, and evidence. These types of projects are commonly referred to as “Focus Groups” and consist of research designs that typically do not use deliberating mock juries (although in some instances they can). In Exploratory Research, the key intent is to find out what is meaningful to jurors. As trial teams “breathe their own exhaust” during months of trial preparation it becomes more and more difficult to ascertain what jurors actually care about versus what “seems important,” and since the latter body of information is typically much larger than the former, these types of projects ultimately serve as a form of information reduction – a means to reduce the elements of the case theory into a more manageable body of facts, arguments and issues that is more streamlined and more effective. The second category – Confirmatory Research – is intended to provide more specific and definitive information that can be relied on to reflect actual trial conditions, i.e., to confirm or disconfirm those particular case issues, themes, arguments and evidence that “drive” ultimate verdict and damages decisions by jurors. From the present perspective, then, Confirmatory Research is more formal, trial- like, and rigorous in implementation, most frequently taking the form of Mock Trial or Trial Simulation research. In these designs, judge’s instructions are used with verdict form interrogatories to charge deliberating juries. It is generally agreed that Confirmatory Research has a greater potential to be predictive of actual trial outcomes compared to Exploratory Research. The intent of the present discussion is to consider alternative formats for Exploratory Research, which might loosely be conceptualized as an examination of “different types of Focus Groups.” While the concept of “Focus Groups” is typically utilized to refer to relatively simple designs in which a group of test respondents is subjected to presentations designed to “throw it all out there and see what sticks,” we will also be considering more specific design alternatives that are implemented to achieve various tactical objectives in the overall preparation of a case. Traditional Focus Groups Typically, the traditional Focus Group design consists of the following steps: 1. Recruit a suitable sample of jury-eligible test respondents who are representative of the trial venire; 2. Develop presentations that summarize the positions of the parties in the case (these presentations may or may not include the case demonstrative evidence and witness testimony); and, 3. Implementation, consisting of a) administration of a pre-test measure designed to assess pre- existing characteristics of respondents; b) subjecting the research participants to the
9. 9 in advance to have mock jurors write about the witnesses in terms of comments or miscellaneous perceptions, possibly including as well standard response scale items measuring selected attributes or credibility dimensions as noted previously. 6. Use the findings for each witness as a foundation for final witness preparation sessions grounded in the perceptions reported by jurors. Conclusions In many instances the term “what sticks” has been used to refer to those themes that resonate with jurors and which are emotionally or perceptually relevant to them. In more formal terms, these are the issues or themes that jurors store and retain in memory for problem-solving the case. Psychologists emphasize that memory does not act like a recording device, but rather is selective, with retention depending on the meaningfulness of the issue or theme to the perceiver. These considerations are vital to the litigator since jurors do not deliberate based on what happens in real time – they deliberate based on what has been stored and retained in memory. Thus, the use of instant feedback devices is not recommended for Focus Group research, because it literally provides too much information. As jurors selectively store in memory only that which is meaningful, that which has become stored can be counted on to reflect the truly dispositive elements of the case, and conversely, that which is not stored in memory is just as useful to the litigator as information that was never presented at all . Therefore, recording of real time reactions to information that never makes it into memory serves up data that essentially has no tactical importance to the litigator. Limiting research to the examination of only that which jurors have kept in their memory to the end, however, represents not only a more parsimonious approach to the perception of the case, but also a more effective one. As stated previously, the more trial-like the procedure, the more one can count on the associated results to mirror trial outcomes – i.e., be predictive. Since prediction leads to control, the importance of making a research project as trial-like as possible becomes self-evident. Thus, studying what jurors have retained at the end mirrors actual trial conditions in which jurors are only using that which they have retained at the end. Focus Groups have an unmistakable “anyone can do that” aura about them and indeed many people literally believe this. Make no mistake: Focus Group research is psychological research, which is unusually sensitive to insidious forms of bias that can easily contaminate the results. Research has shown repeatedly that data in psychological research is readily affected by the wishes and intentions of the researchers. When such wishes and intentions have strong emotional components, the results are even more vulnerable to bias. For example, in our examinations of the accuracy of pre-trial research conducted by both plaintiff and defense trial teams, we generally find that research conducted by the former produce higher damages than research conducted by the latter – often much higher. It is not surprising that the wishes and
7. 7 Thus, while the projects appear to be typical Focus Groups on the surface, the real intent is the derivation of reliable juror profiles for use in jury selection through statistical correlational analyses. However, these projects are extremely productive in that they provide both the profiles and the various substantive findings that become available in the utilization of multiple focus sessions. Special Designs – Witness Effectiveness Training with Jurors Witnesses, of course, are where the “checks” written in opening statements get (or fail to get) “cashed.” While in the early days of litigation consulting it was often stated that “Jurors make up their minds in opening statements,” in fact post-trial juror interviews reveal instead that more frequently, jurors make up their minds while watching the witnesses . In any event, the importance of effective witness preparation is not a topic that engenders debate. Witness preparation most frequently takes the form of some type of “sit down” in which an exchange of performance guidelines, principles, ideas and themes is exchanged between the trial team (lawyers and perhaps consultants) and a witness. While much has been written on these types of preparation activities, in the present context, we convert the notion of witness training into an exercise involving test respondents in a Focus Group setting who provide feedback to the witness. We sometimes refer to this as “industrial strength witness training” – that is, Witness Effectiveness Training with Jurors . Indeed, since jurors do not know that the witness is being provided with their feedback, uncensored comments by jurors may create a degree of immediacy and raw impact that can render the process unsuitable for some witnesses: The types of comments provided by jurors can be difficult to endure for sensitive witnesses, and care should be utilized in choosing the right circumstances in which to use this very powerful witness preparation tool. (By contrast, it is particularly effective for intractable witnesses who are resistant to taking advice from lawyers and consultants). Our experience suggests that the a priori evaluation of witnesses – that is, the assessment of their likeability and credibility without a “reality check” from jurors – is notoriously fallible, even for seasoned litigators and experienced consultants. In other words, nowhere in the field of litigation will a trial team find that an “opinion” is less useful than in determining in advance how a witness will play (except in the case of very good witnesses with a “Our experience suggests that the a priori evaluation of witnesses – that is, the assessment of their likeability and credibility without a ‘reality check’ from jurors – is notoriously fallible, even for seasoned litigators and experienced consultants.”
- 447 Total Views
- 328 Website Views
- 119 Embedded Views
- Social Shares
- 0 Likes
- 0 Dislikes
- 0 Comments
- 0 Facebook
- 0 Twitter
- 0 Google+
- 3 126.96.36.199
- 2 www.courtroomsciences.com
- 1 www.csi-exch01.courtroomsciences.com
- 2 courtroomscience.com
- 1 avenel.courtroomsciences.com
- 2 smtp.courtroomsciences.com
- 2 www.smtp.courtroomsciences.com
- 2 sawgrass.courtroomsciences.com
- 2 dunbar.courtroomsciences.com
Understanding How Jurors Would Value Your Matter854 Views .
Winning the Environmental Case750 Views .
Derailing the Reptile Safety Rule Attack717 Views .
The Generation X and Y Factors [For the Defense]672 Views .
A Psychopath in the Courtroom647 Views .
Assessing Sympathy in Voir Dire620 Views .
Generations X and Y563 Views .
Social Influence in the Courtroom556 Views .