Pseudoscience, “Moneyball,” and Luck

Orin Kerr of The Volokh Conspiracy endorses the following clap-trap, uttered by Michael Lewis (author of Liar’s Poker and Moneyball) in the course of a commencement speech at Princeton University:

A few years ago, just a few blocks from my home, a pair of researchers in the Cal psychology department staged an experiment. They began by grabbing students, as lab rats. Then they broke the students into teams, segregated by sex. Three men, or three women, per team. Then they put these teams of three into a room, and arbitrarily assigned one of the three to act as leader. Then they gave them some complicated moral problem to solve: say what should be done about academic cheating, or how to regulate drinking on campus.

Exactly 30 minutes into the problem-solving the researchers interrupted each group. They entered the room bearing a plate of cookies. Four cookies. The team consisted of three people, but there were these four cookies. Every team member obviously got one cookie, but that left a fourth cookie, just sitting there. It should have been awkward. But it wasn’t. With incredible consistency the person arbitrarily appointed leader of the group grabbed the fourth cookie, and ate it. Not only ate it, but ate it with gusto: lips smacking, mouth open, drool at the corners of their mouths. In the end all that was left of the extra cookie were crumbs on the leader’s shirt.

This leader had performed no special task. He had no special virtue. He’d been chosen at random, 30 minutes earlier. His status was nothing but luck. But it still left him with the sense that the cookie should be his.

So far, sort of okay. But then:

This experiment helps to explain Wall Street bonuses and CEO pay, and I’m sure lots of other human behavior. But it also is relevant to new graduates of Princeton University. In a general sort of way you have been appointed the leader of the group. Your appointment may not be entirely arbitrary. But you must sense its arbitrary aspect: you are the lucky few. Lucky in your parents, lucky in your country, lucky that a place like Princeton exists that can take in lucky people, introduce them to other lucky people, and increase their chances of becoming even luckier. Lucky that you live in the richest society the world has ever seen, in a time when no one actually expects you to sacrifice your interests to anything.

All of you have been faced with the extra cookie. All of you will be faced with many more of them. In time you will find it easy to assume that you deserve the extra cookie. For all I know, you may. But you’ll be happier, and the world will be better off, if you at least pretend that you don’t.

Never forget: In the nation’s service. In the service of all nations.

Thank you.

And good luck.

I am unsurprised by Kerr’s endorsement of Lewis’s loose logic, given Kerr’s rather lackadaisical attitude toward the Constitution (e.g., this post).

Well, what could be wrong with the experiment or Lewis’s interpretation of it? The cookie experiment does not mean what Lewis thinks it means. It is like the Candle Problem in that Lewis  draws conclusions that are unwarranted by the particular conditions of the experiment. And those conditions are so artificial as to be inapplicable to real situations. Thus:

1. The  teams and their leaders were chosen randomly. Businesses, governments, universities, and other voluntary organizations do not operate that way. Members choose themselves. Leaders (in business, at least) are either self-chosen (if they are owners) or chosen by higher-ups on the basis of past performance and what it says (imperfectly) about future performance.

2. Because managers of businesses are not arbitrarily chosen, there is no analogy to the team leaders in the experiment, who were arbitrarily chosen and who arbitrarily consumed the fourth cookie. For one thing, if a manager reaps a greater reward than his employees, that is because the higher-ups value the manager’s contributions more than those of his employees. That is an unsurprising relationship, when you think about it, but it bears no resemblance to the case of a randomly chosen team with a randomly chosen leader.

3. Being the beneficiary of some amount of luck in one’s genetic and environmental inheritance does not negate the fact that one must do something with that luck to reap material rewards. The “extra cookie,” as I have said, is generally produced and earned, not simply put on a plate to be gobbled. If a person earns more cookies because he is more productive, and if he is more productive (in part) because of his genetic and environmental inheritance, that person’s great earning power (over the long haul) is based on the value of what he produces. He does not take from others (as Lewis implies), nor does he owe to others a share of what he earns (as Lewis implies).

Just to drive home the point about Lewis’s cluelessness, I will address his book Moneyball, from which a popular film of the same name was derived. This is Amazon.com‘s review of the book:

Billy Beane, general manager of MLB’s Oakland A’s and protagonist of Michael Lewis’s Moneyball, had a problem: how to win in the Major Leagues with a budget that’s smaller than that of nearly every other team. Conventional wisdom long held that big name, highly athletic hitters and young pitchers with rocket arms were the ticket to success. But Beane and his staff, buoyed by massive amounts of carefully interpreted statistical data, believed that wins could be had by more affordable methods such as hitters with high on-base percentage and pitchers who get lots of ground outs. Given this information and a tight budget, Beane defied tradition and his own scouting department to build winning teams of young affordable players and inexpensive castoff veterans.

Lewis was in the room with the A’s top management as they spent the summer of 2002 adding and subtracting players and he provides outstanding play-by-play…. Lewis, one of the top nonfiction writers of his era (Liar’s Poker, The New New Thing), offers highly accessible explanations of baseball stats and his roadmap of Beane’s economic approach makes Moneyball an appealing reading experience for business people and sports fans alike.

The only problems with Moneyball are (a) its essential inaccuracy and (b) its incompleteness as an analysis of success in baseball.

On the first point, “moneyball” did not start with Billy Beane and the Oakland A’s, and it is not what it is made out to be. Enter Eric Walker, the subject and author of “The Forgotten Man of Moneyball, Part 1,” and “The Forgotten Man of Moneyball, Part 2,” published October 7, 2009, on a site at deadspin.com. (On the site’s home page, the title bar displays the following: Deadspin, Sports News without Access, Favor, or Discretion.) Walker’s recollections merit extensive quotation:

…[W]ho am I, and why would I be considered some sort of expert on moneyball? Perhaps you recognized my name; more likely, though, you didn’t. Though it is hard to say this without an appearance of personal petulance, I find it sad that the popular history of what can only be called a revolution in the game leaves out quite a few of the people, the outsiders, who actually drove that revolution.

Anyway, the short-form answer to the question is that I am the fellow who first taught Billy Beane the principles that Lewis later dubbed “moneyball.” For the long-form answer, we ripple-dissolve back in time …

. . . to San Francisco in 1975, where the news media are reporting, often and at length, on the supposed near-certainty that the Giants will be sold and moved. There sit I, a man no longer young but not yet middle-aged, a man who has not been to a baseball game — or followed the sport — for probably over two decades….

With my lady, also a baseball fan of old, I go to a game. We have a great time; we go to more games, have more great times. I am becoming enthused. But I am considering and wondering — wondering about the mechanisms of run scoring, things like the relative value of average versus power…. I go to the San Francisco main library, looking for books that in some way actually analyze baseball. I find one. One. But what a one.

If this were instead Reader’s Digest, my opening of that book would be “The Moment That Changed My Life!” The book was Percentage Baseball, by one Earnshaw Cook, a Johns Hopkins professor who had consulted on the development of the atomic bomb….

…Bill James and some others, who were in high school when Cook was conceiving the many sorts of formulae they would later get famous publicizing in their own works, have had harsh things to say about Cook and his work. James, for example, wrote in 1981, “Cook knew everything about statistics and nothing at all about baseball — and for that reason, all of his answers are wrong, all of his methods useless.” That is breathtakingly wrong, and arrogant. Bill James has done an awful lot for analysis, both in promoting the concepts and in original work (most notably a methodology for converting minor-league stats to major-league equivalents). But, as Chili Davis once remarked about Nolan Ryan, “He ain’t God, man.” A modicum of humility and respect is in order…. Cook’s further work, using computer simulations of games to test theory (recorded in his second book, Percentage Baseball and the Computer), was ground-breaking, and it came long before anyone thought to describe what Cook was up to as “sabermetrics” and longer still before anyone emulated it.

…I wanted to get a lot closer to the game than box seats. I had, some years before, been a radio newscaster and telephone-talk host, and I decided to trade on that background. But in a market like the Bay Area, one does not just walk into a major radio station and ask for a job if it has been years since one’s last position; so, I walked into a minor radio station, a little off-the-wall FM outfit, and instantly became their “sports reporter”; unsalaried, but eligible for press credentials from the Giants….

Meanwhile, however, I was constantly working on expanding Cook’s work in various ways, trying to develop more-practical methods of applying his, and in time my, ideas….

When I felt I had my principles in a practical, usable condition, I started nagging the Giants about their using the techniques. At first, it was a very tough slog; in those days — this would be 1979 or so, well before Bill James’ Abstracts were more than a few hundred mimeographed copies -– even the basic concepts were unknown, and, to old baseball men, they were very, very weird ideas….

In early 1981, as a demonstration, I gave the Giants an extensive analysis of their organization; taking a great risk, I included predictions for the coming season. I have that very document beside me now as I type…. I was, despite the relative crudeness of the methodology in those days, a winner: 440 runs projected, 427 scored; ERA projected, 3.35, ERA achieved, 3.28; errors projected, 103, actual errors committed, 102; and, bottom line, projected wins, 57, actual wins 56….

By this time, I had taken a big step up as a broadcaster, moving from that inconsequential little station to KQED, the NPR outlet in San Francisco, whence I would eventually be syndicated by satellite to 20 NPR affiliates across the country, about half in major markets.

As a first consequence of that move, a book editor who had heard the daily module while driving to work and thought it interesting approached me with a proposal that I write a book in the general style of my broadcasts. I began work in the fall of 1981, and the book, The Sinister First Baseman and Other Observations, was published in 1982, to excellent reviews and nearly no sales. Frank Robinson, then the Giants’ manager and a man I had come to know tolerably well, was kind enough to provide the Foreword for the book, which was a diverse collection of baseball essays….

At any rate, there I was, finally on contract with a major-league ball club, the Giants, but in a dubious situation…. I did persuade them to trade Gary Lavelle to the Blue Jays, but instead of names like John Cerutti and Jimmy Key, whom I had suggested, Haller got Jim Gott, who gave the Giants one good year as a starter and two forgettable years in the pen, plus two guys who never made the majors. But deals for Ken Oberkfell and especially for John Tudor, which I lobbied for intensely, didn’t get made (Haller called 20 minutes too late to get Oberkfell). I still remember then-Giants owner Bob Lurie, when I was actually admitted to the Brain Trust sanctum on trade-deadline day, saying around his cigar, “What’s all this about John Tudor?” (Tudor, then openly available, had a high AL ERA because he was a lefty in Fenway — this was well before “splits” and “park effects” were commonplace concepts — and I tried to explain all that, but no dice; Tudor went on to an NL ERA of 2.66 over seven seasons.)

When Robinson was fired by the Giants, I knew that owing to guilt by association (remember, Robby wrote the Foreword to my book) I would soon be gone, and so I was. My term as a consultant with the Giants was about half a season. In that brief term, I had had some input into a few decisions, but most of what I advocated, while listened to, was never acted on.

But having once crossed the major-league threshold, I was not about to sink back into oblivion. Across the Bay was an organization with a famously more forward-looking front office, with which I had already had contact. I asked, they answered, and so my career with the A’s began.

Modern analysis has shown a whole treasure chest of interesting and often useful performance metrics, but it remains so that the bedrock principle of classic analysis is simple: out-making controls scoring. What I call “classic” analysis is the principles that I presented to the Oakland Athletics in the early 1980s, which governed their thinking through 20 or so successful seasons, and which were dubbed “moneyball” by Michael Lewis in his book of that title. Because of that book, there has arisen a belief that whatever the A’s do is, by definition, “moneyball”; with the decline in their fortunes in recent years has come a corresponding belief that “moneyball” is in decline — dead, some would say [1] — because the A’s and moneyball are seen as essentially one thing.

That is simply wrong…. “Moneyball,” as the name says, is about seeking undervalued commodities [emphasis added]. In my day, what I regard as the crucial aspects of run-generation, notably on-base percentage, were seriously undervalued, so “moneyball” consisted in finding batters with those skills.

A team that today sustains one of the lowest on-base percentages in baseball, and actively acquires players with drastically low career on-base numbers, is very obviously practicing a different “moneyball” than that for which it became famed. Today’s A’s, it seems, see the undervalued commodities as “defense and athletic players drafted out of high school” (as a recent article on the organization put it). These are not your father’s A’s. What success their new tack will have remains to be seen (their present fortunes are a transition state); but “moneyball” as practiced today by the A’s seems no longer to have at its core the same analytic principles that then-GM Sandy Alderson and I worked with a quarter-century ago, and that I presented to Billy Beane in that now semi-famous paper ["Winning Baseball"]….

In 1994, Sandy promoted Billy Beane to assistant GM. At the same time, he asked me to prepare an overview of the general principles of analysis for Billy, so that Billy could get in one sitting an idea of the way the organization was looking at talent. In the end, I delivered a report titled “Winning Baseball,” with the subtitle: “An objective, numerical, analytic analysis of the principles and practices involved in the design of a winning baseball team.” The report was 66 pages long; I still grit my teeth whenever I remember that Michael Lewis described it as a “pamphlet [on page 58 of this edition of Moneyball].”…

My goal in that report, which I seem to have met, was to put the ideas — not the detailed principles, just the ideas — forward in simple, clear language and logical order, so that they would be comprehensible by and reasonable to a working front-office executive. Sandy Alderson didn’t need a document like this, then or at the outset, but he was a Harvard-trained attorney; I considered myself to be writing not just to Billy Beane but to any veteran baseball man (which, as it turned out, was just as well)….

Lewis not only demotes “Winning Baseball” to a pamphlet, but also demotes Walker to passing mention on three pages of Moneyball: 58, 62, and 63 (in the paperback edition linked above). Why would Lewis slight and distort Walker’s contributions to “moneyball”? Remember that Lewis is not a scientist, mathematician, or statistician. He is a journalist with a B.A. in art history who happened to work at Salomon Brothers for a few years. I have read his first book, Liar’s Poker. It is obviously the work of a young man with a grievance and a flair for dramatization. Moneyball is obviously the work of a somewhat older man who has honed his flair for dramatization. Do not mistake it for a rigorous analysis of the origins and effectiveness of “moneyball.”

Just how effective was “moneyball,” as it was practiced by the Oakland Athletics? There is evidence to suggest that it was quite effective. For example:


Sources and notes: Team won-lost records are from Baseball-Reference.com. Estimates of team payrolls are from USA Today’s database of salaries for professional sports teams, which begins in 1988 for major-league baseball (here). The payroll index measures the ratio of each team’s payroll in a given year to the major-league average for the same year.

The more that a team spends on player salaries, the better the team’s record. But payroll accounts for only about 18 percent of the variation in the records of major-league teams during the period 1988-2011. Which means that other factors, taken together, largely determine a team’s record. Among those factors is “moneyball” — the ability to identify, obtain, effectively use, and retain players who are “underpriced” relative to their potential. But the contribution of “moneyball” cannot be teased out of the data because, for one thing, it would be impossible to quantify the extent to which a team actually practices “moneyball.” That said, it is evident that during 1988-2011 the A’s did better than the average team, by the measure of wins per dollar of payroll: Compare the dark green regression line, representing the A’s, with the black regression line, representing all teams.

That is all well and good, but the purpose of a baseball team is not to win a high number of games per dollar of payroll; it is to win — period. By that measure, the A’s of the Alderson-Beane “moneyball” era have been successful, at times, but not uniquely so:


Source: Derived from Baseball-Reference.com.

The sometimes brilliant record of the Athletics franchise during 1901-1950 is owed to one man: Cornelius McGillicuddy (1862-1956). And the often dismal record of the franchise during 1901-1950 is owed to one man: the same Cornelius McGillicuddy. True fans of baseball (and collectors of trivia) know Cornelius McGillicuddy as Connie Mack, or more commonly as Mr. Mack. The latter is an honorific bestowed on Mack because of his dignified mien and distinguished career in baseball: catcher from 1886 to 1896; manager of the Pittsburgh Pirates from 1894 to 1896; manager of the Philadelphia Athletics from 1901 to 1950; part owner and then sole owner of the Athletics from 1901 to 1954.  (He is also an ancestor of two political figures who bear his real name and alias: Connie Mack III and Connie Mack IV.)

Mack’s long leadership and ownership of the A’s is important because it points to the reasons for the A’s successes and failures during the fifty years that he led the team from the bench. Here, from Wikipedia, is a story that is familiar to persons who know their baseball history:

[Mack] was widely praised in the newspapers for his intelligent and innovative managing, which earned him the nickname “the Tall Tactician”. He valued intelligence and “baseball smarts”, always looking for educated players. (He traded away Shoeless Joe Jackson despite his talent because of his bad attitude and unintelligent play.[9]) “Better than any other manager, Mack understood and promoted intelligence as an element of excellence.”[10] He wanted men who were self-directed, self-disciplined and self-motivated; his ideal player was Eddie Collins.[11]

“Mack looked for seven things in a young player: physical ability, intelligence, courage, disposition, will power, general alertness and personal habits.”[12]

He also looked for players with quiet and disciplined personal lives, having seen many players destroy themselves and their teams through heavy drinking in his playing days. Mack himself never drank; before the 1910 World Series he asked all his players to “take the pledge” not to drink during the Series. When Topsy Hartsel told Mack he needed a drink the night before the final game, Mack told him to do what he thought best, but in these circumstances “if it was me, I’d die before I took a drink.”[13]

In any event, his managerial style was not tyrannical but easygoing.[14] He never imposed curfews or bed checks, and made the best of what he had; Rube Waddell was the best pitcher and biggest gate attraction of his first decade as A’s manager, so he put up with his drinking and general unreliability for years until it began to bring the team down and the other players asked Mack to get rid of him.[15]

Mack’s strength as a manager was finding the best players, teaching them well and letting them play. “He did not believe that baseball revolved around managerial strategy.”[10] He was “one of the first managers to work on repositioning his fielders” during the game, often directing the outfielders to move left or right, play shallow or deep, by waving his rolled-up scorecard from the bench.[12] After he became well known for doing this, he often passed his instructions to the fielders by way of other players, and simply waved his scorecard as a feint.[16]

*   *   *

Mack saw baseball as a business, and recognized that economic necessity drove the game. He explained to his cousin, Art Dempsey, that “The best thing for a team financially is to be in the running and finish second. If you win, the players all expect raises.” This was one reason he was constantly collecting players, signing almost anyone to a ten-day contract to assess his talent; he was looking ahead to future seasons when his veterans would either retire or hold out for bigger salaries than Mack could give them.

Unlike most baseball owners, Mack had almost no income apart from the A’s, so he was often in financial difficulties. Money problems – the escalation of his best players’ salaries (due both to their success and to competition from the new, well-financed Federal League), combined with a steep drop in attendance due to World War I — led to the gradual dispersal of his second championship team, the 19101914 team, who [sic] he sold, traded, or released over the years 1915–1917. The war hurt the team badly, leaving Mack without the resources to sign valuable players….

All told, the A’s finished dead last in the AL seven years in a row from 1915 to 1921, and would not reach .500 again until 1926. The rebuilt team won back-to-back championships in 1929–1930 over the Cubs and Cardinals, and then lost a rematch with the latter in 1931. As it turned out, these were the last WS titles and pennants the Athletics would win in Philadelphia or for another four decades.

With the onset of the Great Depression, Mack struggled financially again, and was forced to sell the best players from his second great championship team, such as Lefty Grove and Jimmie Foxx, to stay in business. Although Mack wanted to rebuild again and win more championships, he was never able to do so owing to a lack of funds.

Had an earlier Michael Lewis written Moneyball in the 1950s, as a retrospective on Mack’s career as a manager-owner, that Lewis would have said (correctly) that the A’s successes and failures were directly related to (a) the amount of money spent on the team’s payroll, (b) Connie Mack’s character-based criteria for selecting players, and (c) his particular approach to managing players.  That is quite a different story than the one conveyed by the Moneyball written by the real Lewis.

Which version of Moneyball is correct? No one can say for sure. But the powerful evidence of Connie Mack’s long tenure suggests that it takes a combination of the two versions of Moneyball to be truly successful, that is, to post a winning record year after year. It seems that Lewis (inadvertently) jumped to a conclusion about what makes for a successful baseball team — probably because he was struck by the A’s then-recent success and did not look to the A’s history.

In any event, success through luck is not the moral of Moneyball; the moral is success through deliberate effort. But Michael Lewis ignored the moral of his own “masterwork” when he stood before an audience of Princeton graduates and told them that they are merely (or mainly) lucky. How does one graduate from Princeton merely (or mainly) by being lucky? Does it not require the application of one’s genetic talents? Did not most of the graduates of Princeton arrive there, in the first place, because they had applied their genetic talents well during their years in high school or prep school (and even before that)? Is one’s genetic inheritance merely a matter of luck, or is it the somewhat predictable result of the mating of two persons who were not thrown together randomly, but who had a lot in common — including (most likely) high intelligence?

Just as the cookie experiment invoked by Lewis is a load of pseudoscientific hogwash, the left-wing habit of finding luck at the bottom of every achievement is a load of politically correct hogwash. Worse, it is an excuse for punishing success.

Lewis’s peroration on luck is just a variation on a common left-wing theme: Success is merely a matter of luck, so it is the state’s right and duty to redistribute the spoils of luck.

Related posts:
Moral Luck
The Residue of Choice
Can Money Buy Excellence in Baseball?
Inventing “Liberalism”
Randomness Is Over-Rated
Fooled by Non-Randomness
Accountants of the Soul
Rawls Meets Bentham
Social Justice
Positive Liberty vs. Liberty
More Social Justice
Luck-Egalitarianism and Moral Luck
Nature Is Unfair
Elizabeth Warren Is All Wet
Luck and Baseball, One More Time
The Candle Problem: Balderdash Masquerading as Science
More about Luck and Baseball
Barack Channels Princess SummerFall WinterSpring
Obama’s Big Lie