Babe Ruth and the Hot-Hand Hypothesis

According to Wikipedia, the so-called hot-hand fallacy is that “a person who has experienced success with a seemingly random event has a greater chance of further success in additional attempts.” The article continues:

[R]esearchers for many years did not find evidence for a “hot hand” in practice. However, later research has questioned whether the belief is indeed a fallacy. More recent studies using modern statistical analysis have shown that there is evidence for the “hot hand” in some sporting activities.

I won’t repeat the evidence cited in the Wikipedia article, nor will I link to the many studies about the hot-hand effect. You can follow the link and read it all for yourself.

What I will do here is offer an analysis that supports the hot-hand hypothesis, taking Babe Ruth as a case in point. Ruth was a regular position player (non-pitcher) from 1919 through 1934. In that span of 16 seasons he compiled 688 home runs (HR) in 7,649 at-bats (AB) for an overall record of 0.0900 HR/AB. Here are the HR/AB tallies for each of the 16 seasons:

Year HR/AB
1919 0.067
1920 0.118
1921 0.109
1922 0.086
1923 0.079
1924 0.087
1925 0.070
1926 0.095
1927 0.111
1928 0.101
1929 0.092
1930 0.095
1931 0.086
1932 0.090
1933 0.074
1934 0.060

Despite the fame that accrues to Ruth’s 1927 season, when he hit 60 home runs, his best season for HR/AB came in 1920. In 1919, Ruth set a new single-season record with 29 HR. He almost doubled that number in 1920, getting 54 HR in 458 AB for 0.118 HR/AB.

Here’s what that season looks like, in graphical form:

The word for it is “streaky”, which isn’t surprising. That’s the way of most sports. Streaks include not only cold spells but also hot spells. Look at the relatively brief stretches in which Ruth was shut out in the HR department. And look at the relatively long stretches in which he readily exceeded his HR/AB for the season. (For more about the hot and and streakiness, see Brett Green and Jeffrey Zwiebel, “The Hot-Hand Fallacy: Cognitive Mistakes or Equilibrium Adjustments? Evidence from Major League Baseball“, Stanford Graduate School of Business, Working Paper No. 3101, November 2013.)

The same pattern can be inferred from this composite picture of Ruth’s 1919-1934 seasons:

Here’s another way to look at it:

If hitting home runs were a random thing — which they would be if the hot hand were a fallacy — the distribution would be tightly clustered around the mean of 0.0900 HR/AB. Nor would there be a gap between 0 HR/AB and the 0.03 to 0.06 bin. In fact, the gap is wider than that; it goes from 0 to 0.042 HR/AB. When Ruth broke out of a home-run slump, he broke out with a vengeance, because he had the ability to do so.

In other words, Ruth’s hot streaks weren’t luck. They were the sum of his ability and focus (or “flow“); he was “putting it all together”. The flow was broken at times — by a bit of bad luck, a bout of indigestion, a lack of sleep, a hangover, an opponent who “had his number”, etc. But a great athlete like Ruth bounces back and put it all together again and again, until his skills fade to the point that he can’t overcome his infirmities by waiting for his opponents to make mistakes.

The hot hand is the default condition for a great player like a Ruth or a Cobb. The cold hand is the exception until the great player’s skills finally wither. And there’s no sharp dividing line between the likes of Cobb and Ruth and lesser mortals. Anyone who has the ability to play a sport at a professional level (and many an amateur, too) will play with a hot hand from time to time.

The hot hand isn’t a fallacy or a matter of pure luck (or randomness). It’s an artifact of skill.


Related posts:
Flow
Fooled by Non-Randomness
Randomness Is Over-Rated
Luck and Baseball, One More Time
Pseudoscience, “Moneyball,” and Luck
Ty Cobb and the State of Science
The American League’s Greatest Hitters: III

Great (Batting) Performances

The normal values of batting average (BA), slugging percentage (SLG), and on-base plus slugging (OPS) have fluctuated over time:

Average major league batting statistics_1901-2015

In sum, no two seasons are alike, and some are vastly different from others. To level the playing field (pun intended), I did the following:

  • Compiled single-season BA, SLG, and OPS data for all full-time batters (those with enough times at bat in a season to qualify for the batting title) from 1901 through 2015 — a total of 14,067 player-seasons. (Source: the Play Index at Baseball-Reference.com.)
  • Normalized (“normed”) each season’s batting statistics to account for inter-seasonal differences. For example, a batter whose BA in 1901 was .272 — the overall average for that year — is credited with the same average as a batter whose BA in 1902 was .267 — the overall average for that year.
  • Ranked the normed values of BA, SLG, and OPS for those 14,067 player-seasons.

I then sorted the rankings to find the top 25 player-seasons in each category:

Top-25 single-season offensive records

I present all three statistics because they represent different aspects of offensive prowess. BA was the most important of the three statistics until the advent of the “lively ball” era in 1919. Accordingly, the BA list is dominated by seasons played before that era, when the name of the game was “small ball.” The SLG and OPS lists are of course dominated by seasons played in the lively ball era.

Several seasons compiled by Barry Bonds and Mark McGwire showed up in the top-25 lists that I presented in an earlier post. I have expunged those seasons because of the dubious nature of Bonds’s and McGwire’s achievements.

The preceding two paragraphs lead to the question of the commensurability (or lack thereof) of cross-temporal statistics. This is from the earlier post:

There are many variations in the conditions of play that have resulted in significant changes in offensive statistics. Among those changes are the use of cleaner and more tightly wound baseballs, the advent of night baseball, better lighting for night games, bigger gloves, lighter bats, bigger and stronger players, the expansion of the major leagues in fits and starts, the size of the strike zone, the height of the pitching mound, and — last but far from least in this list — the integration of black and Hispanic players into major league baseball. In addition to these structural variations, there are others that mitigate against the commensurability of statistics over time; for example, the rise and decline of each player’s skills, the skills of teammates (which can boost or depress a player’s performance), the characteristics of a player’s home ballpark (where players generally play half their games), and the skills of the opposing players who are encountered over the course of a career.

Despite all of these obstacles to commensurability, the urge to evaluate the relative performance of players from different teams, leagues, seasons, and eras is irrepressible. Baseball-Reference.com is rife with such evaluations; the Society for American Baseball Research (SABR) revels in them; many books offer them (e.g., this one); and I have succumbed to the urge more than once.

It is one thing to have fun with numbers. It is quite another thing to ascribe meanings to them that they cannot support.

And yet, it seems right that the top 25 seasons should include so many of Ty Cobb’s, Babe Ruth’s, and of their great contemporaries Jimmie Foxx, Lou Gehrig, Rogers Hornsby, Shoeless Joe Jackson, Nap Lajoie, Tris Speaker, George Sisler, and Honus Wagner. It signifies the greatness of the later players who join them on the lists: Hank Aaron, George Brett, Rod Carew, Roberto Clemente, Mickey Mantle, Willie McCovey, Stan Musial, Frank Thomas, and Ted Williams.

Cobb’s dominance of the BA leader-board merits special attention. Cobb holds 9 of the top 19 slots on the BA list. That’s an artifact of his reign as the American League’s leading hitter in 12 of the 13 seasons from 1907 through 1919. But there was more to Cobb than just “hitting it where they ain’t.” Cobb probably was the most exciting ball player of all time, because he was much more than a hitting machine.

Charles Leershen offers chapter and verse about Cobb’s prowess in his book Ty Cobb: A Terrible Beauty. Here are excerpts of Leershen’s speech “Who Was Ty Cobb? The History We Know That’s Wrong,” which is based on his book:

When Cobb made it to first—which he did more often than anyone else; he had three seasons in which he batted over .400—the fun had just begun. He understood the rhythms of the game and he constantly fooled around with them, keeping everyone nervous and off balance. The sportswriters called it “psychological baseball.” His stated intention was to be a “mental hazard for the opposition,” and he did this by hopping around in the batter’s box—constantly changing his stance as the pitcher released the ball—and then, when he got on base, hopping around some more, chattering, making false starts, limping around and feigning injury, and running when it was least expected. He still holds the record for stealing home, doing so 54 times. He once stole second, third, and home on three consecutive pitches, and another time turned a tap back to the pitcher into an inside-the-park home run.

“The greatness of Ty Cobb was something that had to be seen,” George Sisler said, “and to see him was to remember him forever.” Cobb often admitted that he was not a natural, the way Shoeless Joe Jackson was; he worked hard to turn himself into a ballplayer. He had nine styles of slides in his repertoire: the hook, the fade-away, the straight-ahead, the short or swoop slide (“which I invented because of my small ankles”), the head-first, the Chicago slide (referred to by him but never explained), the first-base slide, the home-plate slide, and the cuttle-fish slide—so named because he purposely sprayed dirt with his spikes the way squid-like creatures squirt ink. Coming in, he would watch the infielder’s eyes to determine which slide to employ.

There’s a lot more in the book, which I urge you to read — especially if you’re a baseball fan who appreciates snappy prose and documented statements (as opposed to the myths that have grown up around Cobb).

Cobb’s unparalleled greatness was still fresh in the minds of baseball people in 1936, when the first inductees to baseball’s Hall of Fame were elected. It was Cobb — not Babe Ruth — who received the most votes among the five players selected for membership in the Hall.

Ty Cobb and the State of Science

This post was inspired by “Layman’s Guide to Understanding Scientific Research” at bluebird of bitterness.

The thing about history is that it’s chock full of lies. Well, a lot of the lies are just incomplete statements of the truth. Think of history as an artificially smooth surface, where gaps in knowledge have been filled by assumptions and guesses, and where facts that don’t match the surrounding terrain have been sanded down. Charles Leershen offers an excellent example of the lies that became “history” in his essay “Who Was Ty Cobb? The History We Know That’s Wrong.” (I’m now reading the book on which the essay is based, and it tells the same tale, at length.)

Science is much like history in its illusory certainty. Stand back from things far enough and you see a smooth, mathematical relationship. Look closer, however, and you find rough patches. A classic example is found in physics, where the big picture of general relativity doesn’t mesh with the small picture of quantum mechanics.

Science is based on guesses, also known as hypotheses. The guesses are usually informed by observation, but they are guesses nonetheless. Even when a guess has been lent credence by tests and observations, it only becomes a theory — a working model of a limited aspect of physical reality. A theory is never proven; it can only be disproved.

Science, in other words, is never “settled.” Napoleon is supposed to have said “What is history but a fable agreed upon?” It seems, increasingly, that so-called scientific facts are nothing but a fable that some agree upon because they wish to use those “facts” as a weapon with which to advance their careers and political agendas. Or they simply wish to align themselves with the majority, just as Barack Obama’s popularity soared (for a few months) after he was re-elected.

*     *     *

Related reading:

Wikipedia, “Replication Crisis

John P.A. Ionnidis, “Why Most Published Research Findings Are False,” PLOS Medicine, August 30, 2005

Liberty Corner, “Science’s Anti-Scientific Bent,” April 12, 2006

Politics & Prosperity, “Modeling Is Not Science,” April 8, 2009

Politics & Prosperity, “Physics Envy,” May 26, 2010

Politics & Prosperity, “Demystifying Science,” October 5, 2011 (also see the long list of related posts at the bottom)

Politics & Prosperity, “The Science Is Settled,” May 25, 2014

Politics & Prosperity, “The Limits of Science, Illustrated by Scientists,” July 28, 2014

Steven E. Koonin, “Climate Science Is Not Settled,” WSJ.com, September 19, 2014

Joel Achenbach, “No, Science’s Reproducibility Problem Is Not Limited to Psychology,” The Washington Post, August 28, 2015

William A. Wilson, “Scientific Regress,” First Things, May 2016

Jonah Goldberg, “Who Are the Real Deniers of Science?AEI.org, May 20, 2016

Steven Hayward, “The Crisis of Scientific Credibility,” Power Line, May 25, 2016

There’s a lot more here.