Off by Only $24,900,000
Five-year old Valerie Lakey is horribly and permanently injured when the water recirculation system traps her at the bottom of a wading pool. Her medical care, including as many as a dozen surgeries and a long line of specialists, will cost millions of dollars, not to mention the loss of any chance for a normal life. Her parents file suit on her behalf against various defendants, including the manufacturer of the missing drain cover.
Shortly before trial, the plaintiffs settle with all but one the defendants for $5.9 million. The plaintiffs demand $4.7 million from Sta-Rite, the manufacturer of the defective drain cover. Despite having $22.5 million of insurance coverage, Sta-Rite offers only $100,000. The case goes to trial. Three weeks later, the jury awards the plaintiff $25 million before the punitive damages phase. At this point, Sta-Rite agrees to settle for the jury verdict.2
Both Sides Miss the Mark
A homeowners association alleges that a real estate broker/developer violated the state's consumer fraud statute and made negligent misrepresentations concerning units at a condominium development. The plaintiff claims that the broker breached his fiduciary as a member of the association board by failing to disclose what he knew about deficiencies in the design and construction of the exterior walls of the building and about the use of improper fill (stumps and construction debris).
The defendant's final offer to settle is $25,000. Plaintiff says it will go away for $300,000. The jury returns a verdict for $3,300,000.
Legal Malpractice Compounded by Settlement Malpractice
The defense rejects the plaintiff’s $325,000 settlement proposal, waiting until the day before trial to make a $50,000 counteroffer. Not long thereafter, the jury renders its verdict: $7 million for plaintiff. With interest, the defendant law firm pays $10 million to satisfy the judgment.3
Statistical Studies Say These Cases Were Not Aberrations
In all three cases, at least one side blundered in the settlement phase of the case. In the drain cover case, the defense completely misjudged the case, and the plaintiffs wisely rejected the paltry $100,000 settlement offer. In the suit against the broker, perhaps both sides erred, each appearing to have wildly underestimated the likely verdict.4 And in the legal malpractice drama, each side mispredicted the outcome, but the plaintiff appears to have been in a fog of total denial.
But, you might be saying, these cases are aberrations. Most cases settle at about the right amount of money or, if not, the trial results in a reasonably just result. Courtroom lawyers generally know what they are doing when they settle cases. Or so we might think.
Well, consider the scientific studies conducted by Randall Kiser, published earlier this year in his monumental book: Beyond Right and Wrong.5 Kiser studied over 2,000 cases in California and approximately the same number in New York, conclusively showing that many parties erroneously reject settlement proposals, obtaining significantly worse results at trial. About 61% of plaintiffs and 24% of defendants who reject settlement offers receive worse results after taking the case to trial.6 In such cases, the parties (or their attorneys or both) have made what Kiser calls “decision errors,” rejecting an offer that would have made them financially better off.
On average, plaintiffs who committed a decision error received a judgment that was $43,100 less than the last offer they rejected in settlement negotiations. Although defendants made bad decisions less frequently than plaintiffs, when they did, the average differential was a whopping $1,140,000. And those statistics do not include the additional legal fees, out-of-pocket expenses, diverted executive time, and emotional costs associated with continued litigation.
If there is so much discrepancy between rejected settlements and eventual verdicts, perhaps we should ask how close we get to the correct numbers the other 95% of the time—when we reach agreement. Routine cases handled by seasoned practitioners are probably resolved on or around the right number much of the time because experienced practitioners and insurance claims adjusters have a large database of knowledge about how such cases will likely end up after trial. But what about the unusual cases or those in which one or both attorneys have minimal or no experience? How close do we get to the correct amount in such cases?
It’s my view that we get it wrong too often. Plaintiffs accept too little or defendants pay too much. I’ve held that view for years, supporting it with anecdotal evidence in workshops and articles. It was only with the publication of Randall Kiser’s research that my intuition was confirmed with statistical evidence.3
But whether we get it wrong 50% or only 25% of the time, getting it wrong is unacceptable.7 The first step to solving the problem is to diagnose why it exists. What factors contribute to decision error in the settlement context?
The Main Causes of Decision Error in the Settlement Context8
The possible causes of sub-optimal settlements in any given case are many, ranging from lack of preparation to poor understanding of the case to pigheaded settlement tactics to unrealistic clients to a positional bargaining style to a variety of subconscious emotional drives and more. But many of these are secondary factors—symptoms of more basic problems.
In my view, there are three underlying causes of settlement failure:
Although all three causes of settlement failure relate to and affect each other, the remainder of this article focuses solely on the last: Inaccurate assessments of what a case is really worth.
“The Net Present Expected Financial Value” of the case (NPEFV) is a long mouthful for what the case is worth today given the remaining costs, the remaining contingencies (including the likelihood of a liability finding and collectability of a judgment), and the weighted average of a range of damages the judge or jury might award if they find the defendant liable. In negotiation parlance, if the trial is my Best Alternative to a Negotiated Agreement (BATNA),11 what is my BATNA worth today—not yesterday and not tomorrow, but today.
The research by Randall Kiser suggests that one or the other side misestimates its BATNA even in cases that settle. Why? What causes us to get the number wrong? Why do we offer $50,000 in a case the jury thinks is worth $7,000,000 or $100,000 when the jury ultimately awards $25,000,000? Is there something wrong with us? Or with how we go about determining the financial value?
As I see it, there is both something wrong with how we are wired and with how we determine case value, and the two are related.
To understand the value of our best alternative to a negotiated agreement (BATNA)— typically, the trial—we must use predictive judgment. We make (mostly unconscious) predictive judgments many times throughout every day, continually predicting what we believe is likely to happen at any given moment. To predict is to project what we do 4 know (evidence and experience) onto the future. Predictions are guesses of what the future might be like. Predictions are paradoxical in that they represent a kind of knowledge about something that we do not yet know.
Uncertainty and Complexity
Two basic problems plague our predictive judgments in litigation and other settings: uncertainty and complexity.
Uncertainty is the absence of information. The more information we have effective use of, the less uncertainty we experience.12 Uncertainty may be the result of (a) ignorance of data we could obtain, (b) unawareness of data we have already gathered but have forgotten, or (c) events that have not yet occurred. The discovery process is designed to reduce the first type of ignorance. Good information collection and processing procedures (file folders, organizational structures, case management software, and the like) help reduce the second. As to the third, we can reduce uncertainty arising from events that have not yet happened in one of two ways: Either we can allow the events to unfold, thereby becoming certain as a result of actual occurrences, or we can organize and analyze the available information, using it to make reasoned guesses about what is likely to happen. This article is concerned primarily with such guesses.
Complexity—too much data—overwhelms our unassisted judgment and decision making processes. Our conscious brains can manage only a limited amount of information at any one time.13 The vast majority of our predictive judgments come about as a result of subconscious processes, commonly called intuition. Our intuition uses recurring patterns to make predictions about what is likely to happen. But it has difficulty with the unfamiliar or highly intricate. Our intuitive processes attempt to force the novel or unknown into existing cognitive templates.14
Our efforts to derive a rational assessment of the financial value of a lawsuit (from each side’s perspective) are confounded by a jumble of factors that should be considered: the likelihood of a liability finding, other dispositive contingencies (e.g., pending motion for summary judgment), non-dispositive contingencies (e.g., whether inflammatory evidence is admitted or excluded), advocacy skills, the jurisdiction, the judge, the range of possible damages, the ability of the defendant to satisfy a judgment, whether the defendant can pay a judgment, and more. There is no way that the unassisted brain can process this mishmash of factors in a rational manner, assessing the numerical probability of each contingency and giving it proper weight with respect to its impact on the ultimate outcome. Without a systematic analytical method to manage all this complexity, therefore, we tend to focus on a few salient factors (key pieces of evidence, the judge’s track record, the highest or lowest reasonable damage award, etc.). And then we settle on the highest (plaintiff) or lowest (defendant) number we can get out of 5 our mouths while keeping a straight face. From there we go through the positional bargaining game, trying to outbluff and outlast the other side.
Most of our predictive judgments are the result of subconscious cognitive systems to which we have no conscious access. Scientists have discovered that these subconscious systems are biased in standard ways and make use of different heuristics—quick and dirty rules of thumb that are good enough most of the time.15 Most of the time, these biases and heuristics serve us well. However, they can occasionally lead to irrational or suboptimal judgments. Three specific cognitive limitations directly affect predictions about case value: the overconfidence bias, the confirmation bias, and the acceptability heuristic.16
Overconfidence. The Greeks called it hubris. In the Middle Ages, it was pride, one of the seven deadly sins. Our less moralistic culture knows it as overconfidence, unbounded optimism, or, in Cordelia Fine’s felicitous phrase, “the vain brain.”17
Making predictions is part of everyday life, saying something like, “I’ll be there in an hour,” and finally arriving 2 hours and 45 minutes later. Or we estimate that a certain phase of a project will be completed in three days that ends up taking three weeks. Or we predict that the Cubs will win the pennant, that the stock market will go up, that the Republicans will win a majority in the mid-term elections, that we will prevail on a motion to dismiss, and on and on—often thoroughly convinced that our predictions, which turn out to be wrong, are perfectly accurate.
For over 30 years, cognitive scientists have studied the overconfidence bias in all kinds of settings, with respect to lay and professional judgments alike. The evidence is conclusive. We have higher opinions of our prediction abilities than we should.
The problem is not limited to everyday affairs; it plagues professionals across the board: physicians and nurses, auditors, college professors, professional traders, investment bankers, political scientists, and . . . drum roll, please . . . lawyers. 18 Experience can reduce the overconfidence bias somewhat—but only with respect to routine cases we see repeatedly over several years: the garden variety rear-ender, the standard breach of contract, the average drug bust. However counter-intuitive, when a case is unusual or outside the lawyer’s experience, her over-confidence actually increases.19
Overconfident predictions occur because we feel we know more than we know (not that we think we know more than we know -- we don't think about it at all). Put differently, we fail to factor in conditions or evidence that we don't see, whether because we have filtered out or we have just not been presented with it. We need a sense of certainty. As professionals, we are supposed to know. Therefore, we come up with a prediction, even when we feel uneasy about doing so.
Confirmation Bias. Our brains continuously build and test hypotheses about the way things are. Once we settle on a belief, however, we like to hang on to it. It’s like the old joke: My mind’s made up; don’t confuse me with facts. “Confirmation Bias” refers to our tendency to seek out evidence that confirms an existing belief, notion, theory, or hypothesis and to neglect contradictory evidence.20 This bias is self confirming in that the more evidence we assemble in support of our belief, the more firmly we hold that belief and the less inclined we are to look for or consider contrary evidence.
“Given the choice between changing their minds and proving there is no need to do so, most people get busy with the proof.” John Kenneth Galbraith
Not only do we subconsciously pick and choose the evidence that supports our existing belief; but we erect barriers to contrary evidence, charging high entrance fees, demanding extraordinarily convincing proof. In the inimitable words of Cordelia Fine, “The brain evades, twists, discounts, misinterprets, even makes up evidence—all so that we can retain that satisfying sense of being in the right. . . . Even the most hastily formed opinion receives undeserved protection from revision.”21
We become wedded to our beliefs, perhaps because they are our beliefs. When we have a dog in the fight, this tendency becomes even more pronounced. We become partisan perceivers, assiduously selecting only that which confirms what we already believe. All of this takes place subconsciously, to be sure. At a conscious level, we appear to ourselves as . . . well, “fair and balanced.”22
The greater our commitment to a belief, the harder it is to dislodge it with contrary or inconsistent evidence. Festinger’s Law
Litigators are in the business of one-sided case-building, which means that we are sitting ducks for the confirmation bias, unwittingly molding facts to fit our theory of the case. And the confirmation bias contributes to our overconfidence when predicting outcomes.
Acceptability Heuristic. For years, cognitive scientists supposed that the subconscious overconfidence and confirmation biases would be mitigated if the predictor was accountable for her judgment. They called this mitigation the accountability effect. The idea was that the brain would clean up its act if it knew that others would pass judgment on its judgment. Initial research tended to confirm that the scientists were on to something. Knowing that peers or other disinterested people will evaluate our judgments—as is the case, for example, with published court opinions—reduces the impact of overconfidence and confirmation biases. When we know nothing about the views of the audience to whom we are accountable, we tend to engage in careful, The greater our commitment to a belief, the harder it is to dislodge it with contrary or inconsistent evidence. Festinger’s Law “Given the choice between changing their minds and proving there is no need to do so, most people get busy with the proof.” John Kenneth Galbraith 7 balanced analyses and to render opinions in the nature of law school exam answers or law review articles.23
Further research revealed a problem, however. The accountability effect mitigates cognitive biases only when the identity and views of those presumed to be watching are unknown. But when we know both the identity and the preferred outcome of the audience to whom we are accountable—a client, for example—we take the least care with our judgment and often make predictions that conform to the views of the audience.24
It is not that we consciously become sycophants, obsequiously pandering to our clients’ wishful thinking. Not at all. We sincerely believe that we are giving our best, objective, professional opinions. Instead, what happens is that we skip the careful analysis we would perform if we had no client or were unaware of her views. We unwittingly rely on a heuristic—a fast and frugal opinion generator25—to reach the answer. Philip Tetlock calls it “the acceptability heuristic.”26
The Significance of our Cognitive Impediments for Case Valuation
The result of the overconfidence bias is that (1) we misestimate the likely outcomes of lawsuits that do not fall into easily recognizable patterns, having an unjustifiably high level of confidence in the accuracy of our predictions, and (2) we may also think more highly of our own and less highly of our opponent’s competence than is warranted. The confirmation bias exacerbates the effects of the overconfidence bias. We literally do not see any reason to doubt our confidence in our predictions because the evidence available to our conscious and subconscious reasoning processes tends to confirm the judgment we are making. We focus disproportionately on the more salient aspects of the case, neglecting items we may want to forget about. Finally, by subconsciously tailoring our judgment to be acceptable to our client, we erect a fortress against consideration of any contrary evidence or argument.
We are not, however, doomed to irrationality. We can mitigate our cognitive biases and heuristics and produce better predictions of litigation outcomes, thereby obtaining more accurate estimates of the Net Present Expected Financial Values of the case for each side by applying analytical methods to the available information—the knowns and the known unknowns—to obtain a reasoned estimate of case value. What follows is a short summary of the method I developed to reduce uncertainty and to manage complexity in ways that mitigate the impact of cognitive biases and heuristics.
The Four Components of Case Valuation
We can squeeze information out of the law and evidence of any case by disaggregating, organizing, and analyzing the jumble of data we have accumulated. This approach is fundamental to all efforts to measure intangibles.27 For case valuation, this means pouring data into the four components of case valuation:
We need to know how each side assesses the numerical probability that the jury will find for the plaintiff, the range of damages the jury is likely to award, the numerical probability that the plaintiff will win each of the remaining dispositive motions or other contingencies such as collectability of the judgment, 28 and the remaining costs of each side.
Obtaining Inputs for the Four Components
The four components provide a way of managing complexity. But unless the estimates generated for each of the components are reasonably accurate—i.e., based on a rational process of data assessment—they might give a false sense of knowledge. A formula is no better than the values entered. Garbage in equals garbage out. But gold in equals gold out as well. Thus, it pays to pay attention to producing golden estimates for each of the four components. Here are some ways to do that.
1. Tell the Stories
What story do you have to tell? What story will the other side present to the fact finder? By going through a mock closing argument for each side, you make yourself aware of how well each case hangs together—or not. You get a sense of how factually and emotionally compelling each perspective is, how much you truly believe in your client’s version, and whether you can readily rebut the best arguments of the other side. This process primes your thinking, bringing assumptions to the top of your consciousness, testing whether you can readily articulate a powerfully persuasive case. That awareness should make it easier to estimate the numerical probability of a liability finding and the range of damages. But it won’t help much with the dispositive contingency component or for estimating the remaining costs.
2. Analyze the Elements
The second method of case analysis, available to every lawyer, is the systematic examination of three aspects of each element of each cause of action:
This is a checklist method, requiring the legal professional to specify each element and to pull together available information concerning that element.29 Using the checklist technique, we write down in summary form both the pieces of evidence and the arguments that each side can legally use to persuade the judge or jury that it should prevail on that element. This is a tedious task but one that every litigator should complete as part of trial preparation in any event. Why not do it before settlement negotiations as well?
We should consider one additional type of influence on judicial decisions as well: Extraneous factors. We could call them extra-legal or even irrelevant factors. This is everything that has no business affecting the outcome of a lawsuit but that does anyway. The biases and prejudices of judges and juries, strengths and weaknesses of the respective advocates, the attractiveness or repulsiveness of the plaintiff, defendant, or key witnesses, the economic mood prevailing in the jurisdiction, the nature of the case, and similar aspects. With the possible exception of advocacy skills, none of this has a legitimate place in a court of law or equity. The outcome should be based solely on the law and the evidence—but it never is and never has been.
Once you have completed the elements checklist, you can weigh the likely impact of the various factors for each side (evidence, arguments, and extraneous items) and make a predictive judgment about (a) the likelihood that the judge or jury will find the defendant liable to the plaintiff and (b) the range of damages the fact finder is likely to award, if the defendant is found liable.
This is an important achievement, both because you have done something to reduce uncertainty and to manage complexity and because you have a better understanding of the case and where any weaknesses reside. A side benefit is that you now know where you need better evidence or arguments, and can take steps to mitigate the impact of extraneous factors—all important parts of good trial preparation. Another benefit is that you now have detailed data with which to explain your professional judgment to the client or the other side or a mediator.
3. Complete a What-Could-Possibly-Go-Wrong? Checklist
A third useful technique should be part of every litigator’s toolkit: I call it the WhatCould-Possibly-Go-Wrong? Checklist. You can use it at any point in the review process but immediately following the elements analysis may be best. Here’s how it works.
Create a table (hand-drawn or using MS Word or Excel) with four columns. In column one, list everything of significance that could possibly go wrong that might affect the outcome of the case on liability, damages, dispositive contingencies, or costs. In column two, estimate the numerical probability that this mishap might actually occur. In column three, indicate one or more of the four components that will be affected. And in column four, estimate the severity of the impact.
Again, this is a tedious exercise that most of us will want to avoid. But, as with the elements analysis, it can pay dividends if you do it right because it forces you to focus attention on aspects of the case you may have suppressed or have not seen, in effect removing the predictive judgment from the clutches of automatic, subconscious processes and giving it the benefit of conscious thought. Whoever provides the estimates for the four components of case valuation should also do this exercise, the main objective of which is to educate intuition. As with geometry, there is no royal road to getting good estimates for use in case valuation.
4. Poll Your Family, Friends, and Acquaintances
The cognitive biases and heuristics summarized above contribute to a kind of tunnel vision about our cases, making it difficult to see them from a dispassionate perspective. And we fail to notice aspects that long ago ceased having any novel impact on us. To counteract the stale case problem, put fresh, unbiased (or less biased) eyes and minds to work. Here’s how:
This process should help you get a sense of the likely liability determination. You may get some useful information on damages, but damages should not be your focus.
The informal survey will tap into the shared sense of justice in your community and will sometimes indicate the relative degree of outrage (or lack of it) that your respondents feel. This is what you really want to find out. Am I defending or prosecuting a case in which the judge or jury will (a) want to help the plaintiff by requiring the defendant to make up for the damage it caused, (b) want to punish the defendant as well (whether or not punitive damages are legally permissible), (c) find that the defendant acted reasonably, or (d) decide that the plaintiff’s conduct was the main cause of damage.
5. Poll Colleagues Who Have Experience Handling this Kind of Case
In the previous survey, you were trying to get a handle on the lay intuition about liability. Here, you want a more expert view of both liability and damages.
The process is similar but has the following twists.
First, register as a user of an online survey service such as Zoomerang or Survey Monkey. Each has a free version that is adequate for the task.
Second, put your written synopsis into the survey and create four questions:
6. Conduct a Focus Group Study
Done well, focus group studies can be the most cost-effective way available of reducing uncertainty about potential case outcomes. You can conduct your own focus group study for less than $1,000 in out-of-pocket expenses or hire a litigation support firm to help with the process.
If you do it yourself, get David Ball’s book, How to Do Your Own Focus Groups. Whether you go it alone or get expert help, you should read up on the use of focus group studies in litigation.31
As with the informal survey of friends and acquaintances, the primary goal with a focus group study is to learn information bearing on the likely liability finding and any aspects of the case that could have an impact on the amount of damages. This is an opportunity (a) to listen to a mock jury deliberate about brief presentations of the case (in essence, closing arguments) by people playing the roles of lawyers for each side and (b) to ask questions afterward to learn more about what the mock jurors thought was influential in their decision. (They will not know what was going on in their subconscious minds,32 but the rationales they produce will be useful nonetheless.)
Dispositive Contingencies and Costs
With respect to dispositive contingencies (motion to dismiss, motion for summary judgment, motion in limine to exclude key witness testimony, etc.), three methods can be highly useful: First, conduct sufficient legal research to determine what the outcome should be (remembering, of course, that your judgment will be affected by cognitive biases). Second, use Jury Verdict Research, WestLaw’s Case Evaluator, LexisNexis’s Total Litigator, or comparable resources to obtain information on the judge’s track record on similar motions in similar cases. Third, conduct an informal survey of colleagues who have appeared before the judge in question.
To obtain a reasonably accurate estimate of remaining costs on your side of the case, the method is similar to the element analysis checklist. Create a written outline of the categories of all work and out-of-pocket expenses remaining in the case. For each category, determine what work remains to be done and who is likely to do it. For each task and each person working on that task, estimate a range of time it will take to complete it, indicate the hourly rate for each person, and compute the range of cost for that task. (When estimating ranges, I use low, medium, and high numbers and estimate the numerical probability for each number. The resulting computation is called the weighted average.)
Completing this estimate of remaining costs has the side benefit of helping us realize the comparative costs of different tactics, which enables us to make better judgments about the relative costs and potential benefits of engaging in them. For example, absent non-financial considerations, an expensive motion for summary judgment with an ultra-low likelihood of success may not be worth pursuing.
Computing the Net Present Expected Financial Value of the Case
Once you have reliable estimates for the values of each of the four components of case valuation, you can compute the Net Present Expected Financial Value of the Case for each side using the Case Value Analyzer™, a proprietary software tool I developed for my consulting and mediation work and which will soon be available on our website: WinBeforeTrial.com.
The Expected Value of Perfect Information
The summary of the Win Before Trial Method of case valuation given here is useful in large and small cases alike. But not every case justifies the time and expense of using all parts of it. We should keep in mind what Douglas Hubbard calls the Expected Value of Perfect Information.33 We can spend hundreds of thousands—even a billion34— dollars reducing uncertainty about potential litigation outcomes in order to produce reliable estimates of case value. But, of course, it does not make sense to pay more than the expected value of that information. How to determine that value is the subject of another article for another day.