Bayes Ball

Bill James in People magazine

2012-02-13T14:38:00.001-08:00

People, June 3, 1991.

No kidding.

First (May 31, 1982) up was a review of the then-newly published Baseball Abstract in the "Picks and Pans" section.

Then in 1991 a personal profile entitled "Holy R.b.i.—it's Statman!" (note: capitalization of RBI is in the archive) appeared, and can be found here.

-30-

Statistical approaches

2012-01-26T15:52:00.000-08:00

At the Statistics for Experimental Biologists blog, a short article about the four main ways to approach a statistical problem: Bayesian, frequentist, information-theoretic, and likelihood, entitled "Putting the methods you use into context". The article points out that there are overlaps between the four approaches, and ends with the statement "Knowing the big picture allows you to reflect on the methods you use, and ask whether they are appropriate for the task. It is also a useful antidote to some of the dogmatism associated with statistical analyses (you don't have to do something one way just because that's how you saw others do it)."

The article also includes some references and links to peer-reviewed journal articles in PDF form that are (contrary to my previous post) freely available to the hoi polloi. One of those is by Gerd Gigerenzer, and appeared in the Journal of Socio-Economics in 2004. In the article, titled "Mindless Statistics", Gigerenzer defines the "null ritual" as having three steps:

1. set up a statistical null hypothesis, but do not specify your own hypothesis nor any alternative hypothesis,
2. use the 5% significance level for rejecting and accepting the null hypothesis, and
3. always perform this procedure.

Gigerenzer then asks "Why do intelligent people engage in statistical rituals rather than in statistical thinking?" His conclusions are rooted in the ritualistic nature of statistical hypothesis testing, in that it has all of the elements of social rituals. First, there is the repetition of the same action (repeating the test over and over). Second, there is a focus on special numbers and colours -- 0.05 and 0.01, two standard deviations, etc. Third, a ritual incorporates fears about serious sanctions for transgressions; in the academy, the adjudicators are professors, academic advisors, journal editors, and textbook publishers. And finally, the ritual relies on "wishful thinking and delusions that virtually eliminate critical thinking", which in statistical research is the p-value.

I will finish with Gigerenzer's final two paragraphs, quoted here in their entirety:

We know but often forget that the problem of inductive inference has no single solution.
There is no uniformly most powerful test, that is, no method that is best for every problem.
Statistical theory has provided us with a toolbox with effective instruments, which require
judgment about when it is right to use them. When textbooks and curricula begin to teach
the toolbox, students will automatically learn to make judgments. And they will realize that
in many applications, a skilful and transparent descriptive data analysis is sufficient, and
preferable to the application of statistical routines chosen for their complexity and opacity.
Judgment is part of the art of statistics.
To stop the ritual, we also need more guts and nerves.We need some pounds of courage
to cease playing along in this embarrassing game. This may cause friction with editors and
colleagues, but it will in the end help them to enter the dawn of statistical thinking.

-30-

The bizarre world of academic journals

2012-01-25T22:10:00.000-08:00

On a fairly regular basis, for both my work and for light entertainment, I seek out journal articles written by leading scholars. And I begin every search knowing there is a high probability that it will be foiled, and I will find myself unable to access those materials.

Laura McKenna, writing for The Atlantic, sums the bizarre nature of academic journals with this well-written and succinct article titled "Locked in the Ivory Tower: Why JSTOR Imprisons Academic Research".

Highly recommended.

-30-

SABR Analytics Conference

2012-01-13T13:55:00.000-08:00

The Society for American Baseball Research (SABR) has just announced that they will be hosting the first-ever conference dedicated to baseball analytics from March 15-17, 2012.

All of the "featured speakers" shown are team general managers and executives -- but I anticipate that as more speakers are announced there will be some number crunchers added to the list.

Hopefully there will be a conference procedings document produced for those of us who won't be able to attend.

-30-

The music of my mind

2012-01-07T17:18:00.000-08:00

Jason Branon at Baseball Nation offers up "ProGS - The Pronunciation Guide For Sabermetricians", which is just what it says.

At first I thought, why bother? I tended to side with Jason's leader, Rob Neyer -- the only people who read this stuff never leave the house. And then I realized that isn't entirely true -- at the annual SABR convention (in Minneapolis this year), baseball nerds get together, and the quantitative specialists need to be able to argue about the "new stats" and the interpretation thereof, rather than wasting precious time and energy arguing about pronunciation. (Which is a whole other dimension of nerdiness.)

Perhaps someone can create a sabermetric rewrite of the Gershwin brothers' "Let's call the whole thing off" -- the tuh-MAY-toe / ta-MAH-toe song.

Fred shows Ginger how to execute an effective throw from second base on a double play pivot.

-30-

OT: Unemployment Rates in the U.S.A.

2011-12-14T19:21:00.000-08:00

A great example of a truly awful graph was posted on Flowing Data, starting a conversation on The Book.

I posted the following comment there:

Wexler/27 beat me to the BLS data sets... I will note that the "discouraged" numbers can be found in the "characteristics of the unemployed" tables. The different ways of parsing what constitutes "unemployment" are ways to try to get to the nuance in why people aren't working. The narrow "actively seeking work" definition of defining who is in the labour market is a way to cut through demographic changes (e.g. in the post-WWII period, when most women were June Cleavering and not seeking work), increases in post-secondary enrollments, etc.
With that said, the increase in people who have thrown in the towel is, to me, one of the most disturbing parts of the current recession.
Wexler/7 and MGL/8 raise the question of attribution -- how much is the President (in the current circumstance, Obama) responsible for unemployment rates?
I dug up some historical U.S. unemployment rate data going back to 1948, and I have posted a chart of it to my own blog (because I have no idea how to do that directly).
A summary: the current increase in unemployment started in the last year of G.W. Bush's 2nd term (rising from 5.0% in January 2008 to 6.8% by the November, the month of the election). Going back, there was a peak in unemployment during the first G.W. presidency, and another that straddled G.H.W. Bush and Clinton. And the worst unemployment rates since the Great Depression (higher rates and with a longer peak than the current phase) were during the first term of Ronald Reagan's presidency.

Here's a chart of U.S. unemployment rates from January 1948 to November 2011:

And for those of you wanting to focus on the more recent period, the past 20 years (less a month):

I obtained the U.S. Department of Labor data set from the Economic Research pages of the Federal Reserve Bank of St. Louis here, and have converted that text file to an Excel file.

More Bureau of Labor Statistics (BLS) unemployment data can be found here, here and here.

-30-

Farm system success

2011-11-21T10:32:00.000-08:00

Flip Flop Flyball has had a number of good infographics (and humour items) in the past, but their recent "Wins and loses throughout each team's system" chart is particularly interesting. One thing that caught my eye is that no team in the Houston Astros system managed to break the .500 mark in 2011.

This raises a question in my mind. Can the current performance of minor league afflitates be used to predict MLB team performance at some future date? (Economists would call this a leading indicator.) All of the research on minor league performance that I'm aware of is in service of forecasting individual player performance. For good review of that work, see "The Projection Rundown" at Fangraphs.

But I'm wondering if the fluid nature of the minor leagues will yield any sort of meaningful result at the team level. Not only are players constantly moving up and down between the levels, it also seems to me that they are every bit as likely (if not more so) to move mid-season from one organization's farm system to another (resource: Baseball America's listing of minor league players). And the farm teams themselves are prone to shifting from one organization to another, and moving up and down the levels. As one example, the Vancouver Canadians of the single-A Northwest League were affiliates of the Oakland A's for 11 seasons, but in 2011 came under to Toronto Blue Jays umbrella (they finished with a 0.513 record, second in their division).

Which then leads me back to the infographic: other than 2011 results, does it tell us anything?

-30-

Lewis & Beane interview

2011-11-14T14:50:00.000-08:00

Moneyball (the movie) opens in the U.K. on November 25, and as part of the publicity, The Financial Times features an in-depth interview with both Michael Lewis and Billy Beane by Simon Kuper.

It's quite a revealing interview, that digs into the relationship between Lewis and Beane -- why Lewis was interested in finding the story, and why Beane let Lewis hang around.

But to whet your appetite, here's a couple of highlights. First, a quote from Michael Lewis:

"Baseball is a stupid-making enterprise in that nobody wants to be singled out or say something dumb. You wander in the clubhouse and it’s amazing how incurious the players are. One reason I was attracted to Scott Hatteberg [the former A’s player] as a character: he was just curious: ‘What the hell are you doing here, man?’”

On the criticisms of Moneyball:

There are two silly objections often made to Lewis’s book. The first is that if Moneyball works so well, then why haven’t the A’s had a winning season since 2006? We meet on a sunny October morning, mid-playoffs, a perfect day for baseball, but the team’s season has long since ended.

However, the people who make this objection don’t seem to grasp the basic principles of imitation and catch-up. Once all teams are playing Moneyball, then playing Moneyball no longer gives you an edge. Indeed, the richer clubs have the means to play it smarter. The New York Yankees recently hired 21 statisticians, Beane marvels.

The other common snipe is that Beane should never have spilled his secrets to Lewis. That ruined the A’s, the critics say. But Lewis dismisses the charge. First, he notes, Beane had never imagined their conversations would spiral into a book. Lewis says, “I was going to do something little. By the time I thought I was going to do something big I’d hung around so much it would have been socially awkward to ask me to leave.”

Second, notes Lewis, by 2002 Moneyball was already spreading. The book ends with the Red Sox offering Beane the highest GM’s salary in baseball history. Only when Beane turned them down, having decided after Stanford that he’d never do anything just for money again, did the Red Sox hire Epstein. “The market was moving already,” says Lewis. “The teams that wanted to do it were going to do it anyway, so no book was going to make any difference. My view is the only effect of the book was to give them [the A’s] the credit. If no book had been written, Theo would have been branded the man who reinvented baseball.”

Of course, Epstein's stuff worked in the playoffs.

-30-

The Sabermetric bookshelf, #2

2011-11-07T12:06:00.000-08:00

Baseball Analyst, 1982-1989 (Bill James, publisher and editor)

SABR is now hosting -- the the blessing of Bill James, and through the work of Phil Birnbaum -- the complete Baseball Analyst. Between 1982 and 1989, Bill James published 40 issues of Baseball Analyst, which in retrospect is now recognized as the launch pad for some fundamental thinking about using quantitative approaches to understand baseball.

The initial issue got off to a great start, with an article about fielding by Paul Schwarzenbart. In his introduction to the issue, James writes that the article "demonstrates that fielding statistics, like batting and pitching but apparently even more so, are the products in part of circumstances as well as men." This is a topic that, 30 years later, continues to provide plenty of fodder for analysis (e.g. this blog post from a month ago by Tangotiger, "Not all fielding opportunities are created the same").

In later issues, there are articles covering the usual parade of topics: clutch hitting, ballpark effects, how much young pitchers should work, ageing of ball players, and of course movie reviews.

There's also familiar names: Pete Palmer, Phil Birnbaum, and Bill James himself.

All in all, Baseball Analyst is an interesting time capsule. The tools the sabermetric community use to communicate have shifted -- when was the last time you subscribed to a magazine produced on a typewriter and mimeograph? But more importantly, it demonstrates how thinking about these topics has shifted. This shift is both because of further research (we know more than we used to) and because of the proliferation of data and cheap computing power

But it also shows that in spite of 30 years of analysis, there are still many questions unresolved.

-30-

World Series prediction: the Bill James method

2011-10-18T11:57:00.000-07:00

Bill James developed a method for predicting playoff series winners, last updated in the 1984 edition of Baseball Abstract in an essay titled "The World Series Prediction System, Revisited". At that point, it had a pretty good track record -- 73% success in predicting the winner of all the postseason series in the 20th century.

Mike Lynch over at seamheads.com used the method (without any adjustments, updates, or other tweaks) to predict the 2010 World Series -- which correctly identified the Giants.

This year, Lynch has again used the tool and tabulated the Rangers and the Cardinals according to the Bill James method.

The result: the Rangers come out as solid favorites.

(A couple of other older references to previous use of method are here and here. Other than that, I haven't found anything on the web that uses or updates the method.)

-30-

World Series prediction

2011-10-17T19:11:00.000-07:00

The 2011 World Series starts in a couple of days, and it's time for the pundits to come out and make their predictions. Over on coolstandings.com they've posted their prediction for the World Series. Here's a screenshot of their "smart" prediction:

(The "dumb" prediction is 50/50 for either team, so there's no point talking about that. And I've posted a screenshot, since their predictions are live and will change upon the outcome of the first game of the World Series. An example of the Monty Hall problem, in real life.)

To summarize: Texas shows as having a 68.2% probability of winning the World Series.

I'm not sure of the details of their methodology, but we can use each team's regular season win/loss record to employ the "log5" approach to come up with our own prediction. So I did that, and my first prediction is for a Texas victory (58% probability) -- and if pressed to predict the series length, it would be Texas in 6 games (17% of the outcomes are Texas 4-2). Both probabilities are substantially lower than the coolstandings prediction.

But we can be a bit more sophisticated in our approach, using an adjusted win/loss percentage that employs a Bayesian adjustment to each team's final result. (This is the same method I used back in May for the early season results -- after 162 games the impact of the prior is much reduced.) This changes Texas' winning percentage to 0.571, and St. Louis to 0.543. (Google doc spreadsheet here.) Using the log5 formula, this gives the Rangers a 0.538 edge over the Cardinals.

Working through the 7 game series, Texas' probability of winning the World Series is 56%.

And we can be still more clever, by considering the road/home splits of each team.

Team W-L % posterior
-------- ----- ---- ---------
Texas 96-66 .593 .571
- home 52-29 .642 .591
- road 44-37 .543 .527

St Louis 90-72 .556 .543
- home 45-36 .556 .535
- road 45-36 .556 .535

The home-road splits improve things for the Cardinals, since they had a better home record than Texas' road record and thus become more likely to win a home game. As well, the Cardinals have home field advantage (but only on game 7 -- the Rangers have home field advantage in a 5-game series. But I digress.) After using the home-road splits, Texas still remains the favorite, but the probability is down to 54%.

While my approaches still give Texas the biggest likelihood of victory, my estimates are less emphatic than the probabilities over at coolstandings. Based on the characterizations used at coolstandings, my methods lie somewhere between "dumb" and "smart". "Average intelligence", perhaps.

Tow Mater says "Rangers in 6. But I had the Phillies and the Brewers beating the Cardinals, too".

-30-

And on the topic of infographics...

2011-10-08T11:49:00.000-07:00

Source: http://www.flickr.com/photos/smoy/6143338263/, via Andrew Gelman.

-30-

WPA contribution infographic

2011-10-07T12:12:00.000-07:00

I like these WPA word cloud graphics from SB Nation by Kevin Dame, describing the player contributions in last night's Tiger-Yankee ALDS game 5. (From
http://mlb.sbnation.com/2011/10/7/2474640/yankees-tigers-game-5-visual-box-score.)

One of the things I like is that they emphasize that that WPA (Win Probability Added) is circumstantial.

For the Tiger pitching staff, the starter Fister gave up only one run over five innings (that is, four scoreless innnings), but gets a smaller font than the closer Valverde who worked only the scoreless ninth inning. An easy example is Fister worked a 1-2-3 1st innning with a 2-0 lead, which was worth 0.052 WPA. By contrast, Valverde's 1-2-3 9th inning with a one-run lead (3-2 score), was worth 0.222 WPA. Being later in the game and with a tighter score yielded a higher WPA.

And for the Yankee hitters, ARod's strikeout to end the game (the end of Valverde's 1-2-3 ninth) was only one-third as important to the Yankee defeat (-0.053 WPA) as Swisher's strikeout to end the 7th inning, when the bases were loaded (-0.154). Of course, on Swisher's strikeout Tiger pitcher Joaquin Benoit set himself up for the big 0.154 WPA by coming in with a runner on 1st, then giving up two singles to load the bases, followed by a walk to close the lead to one run.

(The Fangraphs box score has the details that were used to make the word clouds, while the individual play log, with the WPA for each at-bat, is here.)

-30-

Actuarial baseball

2011-10-04T13:55:00.000-07:00

Been off the grid for a while...

A couple of weeks ago, Josh Hamilton of the Rangers hit a grand slam that got an above-average amount of attention, since it was tied into a promotion being run by a flooring company. The title of this article describing the homer could instead be "Josh Hamilton's grand slam yields big insurance payout":
http://espn.go.com/dallas/mlb/story/_/id/6971469/texas-rangers-josh-hamilton-slam-triggers-free-flooring-payout

Somebody, somewhere, in some insurance company, sold coverage for this promotion. And that same somebody (we hope) must have sat down and calculated the probability of Hamilton hitting a grand slam over a one month period, and set the premium based on that probability.

Summary: insurance is gambling.

-30-

Bayes book

2011-08-09T13:47:00.000-07:00

I recently learned about a new book by Sharon Bertsch McGrayne, The Theory That Would Not Die: How Bayes' Rule Cracked the Enigma Code, Hunted Down Russian Submarines, and Emerged Triumphant from Two Centuries of Controversy.

I haven't got a copy yet, but based on a couple of reviews, it seems like it's going to be a good read. The review in Significance magazine describes it thus: "At times reading like a historical account, at times like investigative journalism, at yet other times like a statistical commentary."

Other reviews: New York Times Sunday Book Review, Boston Globe, and Nature (subscription required for on-line access).

-30-

Physics of baseball

2011-06-30T13:18:00.000-07:00

In 1990, I read a great little book called The Physics of Baseball by Robin K. Adair (now in its third edition). It's strongly recommended to anyone interested in the subject matter. And here's a short Q&A with Adair at Popular Mechanics from a couple of years ago.

But a few other items of note on this subject have popped up recently.

First, a great chart at The Book, with the speed of the ball off the bat as the X axis and the angle of launch as Y, showing the outcome (from ground ball to home run) at the points X,Y.

Then, there's an article in the latest issue of the American Journal of Physics by Faber, Smith, Nathan, and Russell called "Corked bats, juiced balls, and humidors: The physics of cheating". You can also find a short summary of the article at Smithsonian.com.

There are three questions asked, each with a nuanced answer.

1. Question: "Can a baseball be hit farther with a corked bat?"
Answer: "... while corking may not allow a batter to hit the ball farther, it may well allow a batter to hit the ball solidly more often."

2. Question: "Is the baseball juiced?"
Answer: The researchers "found no evidence that baseballs of today are more or less lively than baseballs used in the late 1970s."

3. Question: "What's the deal with the humidor?" (Or, "is it plausible that the humidor accounts for the decrease in offensive statistics at Coors Field since 2002?"
Answer: Yes.

For those interested in a deeper dive into this topic, one of the co-authors of the study, Alan Nathan, has a page dedicated to the physics of baseball.

-30-

Update 2011-07-13: Tango at The Book posted a link to the Smithsonian article, and there has been plenty of commentary including a number of responses from Alan Nathan.

Two months in, a Bayesian look at the standings

2011-06-01T20:25:00.000-07:00

At the end of April, I posted "Early season standings and Bayes" that took two different approaches to regressing the early season standings to come up with a prediction for the eventual result for the full season.

So here we are at the end of May, and there's been a lot of movement in the standings, so here's an update to the spreadsheet. Although the Phillies and the Indians remain at the top of the standings, they are starting to regress downwards. At the end of April, both teams were "on pace" to win 112 games in the season, but the regression showed a more modest result of 93 wins. A month later both teams are "on pace" to win 100, but the Bayesian approach suggests that they are more likely to win 91 games.

If you are a Twins or an Astros fan, there is no solace in the fact that both teams have not regressed toward the mean over the past month, but have instead continued to play at roughly the same level they exhibited in April. The regression model now predicts the Twins will end up at 65 wins, which would be the lowest in MLB. Of course, this prediction is based only on the team's performance to date -- it doesn't consider the number of injuries the Twins are currently dealing with.

-30-

Ouch

2011-05-29T09:23:00.000-07:00

XKCD on sports. Is there a way to test (in a quantitative manner) the hypothesis that baseball is the worst offender?

-30-

When labour market research goes to the ballpark

2011-05-06T13:15:00.000-07:00

In a recently issued paper called "Productivity, Wages, and Marriage: The Case of Major League Baseball", economists Francesca Cornaglia and Naomi E. Feldman examine the "marriage premium" -- the fact that controlling for other influencing factors, married men earn more than unmarried men. In most situations, a variety of confounding variables muddy the waters -- things like geographic location, differences across occupations, and poor productivity measures. Cornaglia and Feldman innovatively use information from MLB to control for those variables.

Derek Jeter, the exception that proves the rule.

The abstract:

Using a sample of professional baseball players from 1871 - 2007, this paper aims at analyzing a longstanding empirical observation that married men earn significantly more than their single counterparts holding all else equal. There are numerous conflicting explanations, some of which reflect subtle sample selection problems (that is, men who tend to be successful in the workplace or have high potential wage growth also tend to be successful in attracting a spouse) and some of which are causal (that is, marriage does indeed increase productivity for men). Baseball is a unique case study because it has a long history of statistics collection and numerous direct measurements of productivity. Our results show that the marriage premium also holds for baseball players, where married players earn up to 20% more than those who are not married, even after controlling for selection. The results are generally robust only for players in the top third of the ability distribution and post 1975 when changes in the rules that govern wage contracts allowed for players to be valued closer to their true market price. Nonetheless, there do not appear to be clear differences in productivity between married and nonmarried players. We discuss possible reasons why employers may discriminate in favor of married men.

You can hear Dr. Cornaglia discuss the research on the BBC programme More or Less (2011-04-29), starting at roughly 10'50".

My initial reaction regards neither the findings nor the methodology, but the fact that other than a mention of the Lahman database, the list of references does not include any of work of the sabermetric research community. At one point in the discussion of productivity measures the authors write "Most modern-day baseball enthusiasts and commentators consider the latter two statistics [OPS and EqA] to be the most accurate measures of a player’s productivity", but the authors neither refer to any authority to support that statement nor discuss fact that others have critiqued those measures.

This is not the first time that academics have utilized the contributions of the sabermetric community in supporting their research (in this case, it provides a vital element in the foundation of the productivy measure) but then failed to acknowledge that work. For a well-reasoned discussion of that topic, please read Phil Birnbaum's "Chopped liver II".

-30-

Early season standings and Bayes

2011-05-01T11:56:00.000-07:00

Early season performance has been a hot topic this year (not that it isn't a topic of discussion every year). I wrote about it, using a simple approach of assuming that every team is .500, and a more recent addition in the blogosphere is Rob Neyer's take.

Last week Kincaid over at 3-D baseball had great post that used Boston's 2-10 start to go down a detailed and more sophisticated Bayesian path to estimating the team's true talent. Tango posted a link to Kincaid's blog, and added a few details that incorporate actual observations. A key element in this is that the observed spread of talent is wider than the theoretical .500 level of all teams. (If all teams were .500, the random component would result in a standard deviation of 0.039. In reality, the standard deviation is wider, at 0.071 -- the implication of this is that there are real talent differences between teams, with some teams having a true talent level above .500 and others below.)

My modest contribution to this thread is here: a Google doc spreadsheet that show all of the MLB team's current record (as of 2011-04-30), and then takes two different Bayesian-based methods to predict each team's final season outcome.

The first set are the yellow columns, which replicate Kincaid's "shortcut" approach, with the implied regression of 69 games noted by Tango. The blue columns take a different approach that uses the standard deviation of both the observed performance to date and the long-term observations (every MLB team season outcome from 1961-2010) as the prior.

The difference in the result generated between these two approaches is relatively modest. (It's worth noting that the relative position on the standings does not change.) What is apparent is that with roughly 25 games played this season, there are solid differences appearing in the team performances. This method forecasts that Cleveland and Philadelphia will regress downward from .692 (a 112 win season) to .573 and end up with 93 wins. At the bottom of the table, it suggests that the Twins will improve from .346 (58 wins on the season) to .444 and a much more respectable 74 wins over the course of the season.

-30-

Words with meaning

2011-04-28T12:14:00.000-07:00

Article at Slate entitled "Turning words into touchdowns", on the work of Achievement Metrics. This company takes interviews from various players and parses them, and then correlates the pattern with on- and off-field performance. From AM's website:

Our analyses of players’ speech, arrest, and suspension data have shown that differences in players’ speech while in college can predict which players are more likely to exhibit off- and on-field behavioral problems during their professional careers.

I'm not sure whether this would work for baseball, since the players just speak in clichés. (Warning: clip includes profanities.)

-30-

Baseball fans are crazy

2011-04-19T19:01:00.000-07:00

Or so says this ad for Strat-o-matic, posted at If Charlie Parker was a Gunslinger...
(Note: the main Charlie Parker page is not appropriate for at-work viewing.)

(Click the image to enlarge.)

-30-

Kicking at the darkness

2011-04-12T08:43:00.000-07:00

Last night (2011-04-11) the Seattle Mariners pulled off a preposterous comeback, defeating the Blue Jays 8-7 after trailing 0-7 heading into the seventh inning. Other teams have had comebacks from being down by 7 runs, and pulled off comebacks in bigger games. But as Rob Neyer has pointed out, what made this so unexpected and so special was that the Mariners have been, in a word, hapless. The early part of this game was the best/worst example of their struggles.

The FanGraphs plot (chart below) follows what has become a disturbing Mariner trend this year -- the line quickly plummets to the sub-10% win expectancy range in the early innings, and slowly drifts towards zero from there. (Check out the games vs. Cleveland the day before and the home opener on 2011-04-08 for recent examples). This time, after bottoming out at 0.3% when Luis Rodrigez (the game's eventual hero) struck out to lead off the Mariner half of the seventh inning, the WE line zigzagged its way to the other end of the scale.

Blue Jays @ Mariners, 2011-04-11 (source: FanGraphs)

For the Mariners, a second consecutive 100 loss season (which would be the third in four seasons) is not at all out of the question. But for the fans who stuck with it last night, this was one for the ages. Or the U.S.S. Mariner game summary, quoted here in its entirety: "That was horrible, then awesome. Baseball is fun."

The title I used for this entry is a reference to Bruce Cockburn's song "Lovers in a Dangerous Time". In the article linked above, Neyer wrote "It was somebody smart, or maybe an episode of Scrubs, that said nothing worth having comes easy." The song contains the line "Nothing worth having comes without some kind of fight/Got to kick at the darkness until it bleeds daylight". In the late innings of last night's game, the Mariners showed some kick.

-30-

Social mobility toward the mean

2011-04-11T14:06:00.001-07:00

The April 8, 2011 edition of the BBC radio program More or Less* includes a discussion of regression toward the mean in the context of social mobility stats in the U.K. Most of the analysis has focussed on the impact social class has on long-term education outcomes. In particular, much has been made of the fact that the analysis suggests that the low ability children from high social class catch up and pass the high ability children from low social class.

But in the broadcast Daniel Read, professor at Warwick Business School, has offered a critique (link to written version) that points out that the analysis has not accounted for "one of the oldest statistical problems of all" (the BBC's description): regression toward the mean. The source of the problem is correctly identified as the bias introduced by including only the highest and lowest performers in the groups shown in the chart. The children closest to the mean for that social class have been excluded.

Because only the extreme ends of the education outcomes tests of the two social class groups have been selected, the poorest performers naturally show improvements while the higher performers show declines. From the broadcast:

It's not that it's [the differences in outcomes between social class] all fluke. But if there's any element of luck at all -- which there surely is, because we're talking about ability tests for toddlers -- then we have to allow for what we'd expect to happen when that luck fails to last. And what we'd expect to happen is pretty much what the graph in the government's social mobility strategy shows, which is that the next time you test the children all the high performers have dropped off. But especially the poorer kids who, remember, Nick Clegg says were disadvantaged from birth. And all the lower performers have caught up, but especially the richer kids. And then as you continue to test, the richer kids gain on the poorer kids at a very much less dramatic pace.

The easiest way to spot the regression towards the mean? The enormous change from the first to the second measurement, as much of the selection bias at the first measurement point disappears. The high and low performers were selected not on the basis of their long-term outcomes, but on the results of the first test. In subsequent tests the children in these extreme cases will move toward the mean, and closer to their "true talent".

Accounting for regression toward the mean does not mean that social class doesn't have a relationship with education outcomes. But accounting for the regression toward the mean would moderate the magnitude of the difference between the two social classes.

*The linked page has a text summary of the program, a copy of the chart in question, streaming audio of the program, and links to the podcast and supporting documents. The item begins at roughly 17' 25" of the podcast.

-30-

Meaningless numbers

2011-04-10T09:56:00.000-07:00

There's been plenty of chatter on the sabermetric blogs lately about the meaningless stats bandied about by broadcasters during the early stages of the season. The best way I've seen the validity of these numbers debunked is on Jeff Sullivan's post on the Indians-Mariners game at Lookout Landing (on SB Nation):

In the bottom of the first, the broadcast flashed a Justin Masterson [the Indians' starting pitcher] stat graphic showing his lefty/righty splits on the season. After one game. The only thing I wish is that they would've shown his home/road splits instead.

-30-

On pace for a 162 loss season!

2011-04-08T11:03:00.000-07:00

Two days ago I responded to an on-line article about the Orioles' 4-0 start to the season, pointing out that it's not at all surprising -- using the laws of binomial probability -- to see three of the 30 MLB teams at 4-0 to start the season.

What I didn't mention were the teams that started the season on a losing streak. And now, heading into this weekend's play, the 0-6 Red Sox and Rays that are getting the attention of the punditocracy. For the Red Sox, it's the poorest start since the 1945 season, and the Rays haven't ever gone 0-6 to start the season in their comparatively short history.

Some writers have acknowledged the probability and the history of being 0-6: Dave Cameron at FanGraphs writes "Is it time to panic in Boston?", and Cliff Corcoran at S.I.'s piece is "It's still early, but history is against winless Red Sox, Rays and Astros" (which was written before yesterday's games, when the teams were 0-5).

An entirely different view can be found Baseball Prospectus, where Steven Goldman uses the 1987 Brewers, who went 13-0 and then 20-3 (Goldman writes, "on pace for a 141 win season") before hitting a 12 game losing streak, and the opening sequence of Tom Stoppard's existentialist play Rosencrantz And Guildenstern Are Dead to suggest that sometimes things operate outside the laws of probability.

(Watch the scene in question, from the 1990 film with Gary Oldman and Tim Roth as the title characters.)

In the play, the characters are faced with a preposterously long string of coin-flips that land heads. This leads Guildenstern to say "A weaker man might be moved to re-examine his faith, if in nothing else at least in the law of probability."

Even though the two teams are 0-6, the weakness of my faith in the laws of probability is not yet tested.

-30-

Gelman on baseball

2011-04-07T17:31:00.000-07:00

Andrew Gelman has published a few blog articles lately that hit on baseball.

First up, "Bill James and the base-rate fallacy", where he points out a flaw in James' reasoning that arises from the "availability heuristic".

Second, at The Statistics Forum, a comparison of predicting future performance at a significant transition point in "Minor-league Stats Predict Major League Performance, Sarah Palin, and Some Differences Between Baseball and Politics".

I don't have anything to add, other than to say it's encouraging to see one of the best statistical thinkers in the academy using baseball as a point of reference.

-30-

On pace for a 162 win season!

2011-04-06T17:28:00.000-07:00

Only at the beginning of the season would a four-game winning streak get you an article on S.I.

The probability of a true .500 team playing four games against other true .500 teams and winning them all is 6.25% -- or roughly 2 out of 30. In other words, at this point in the season we could realistically expect two teams to be at 4-0. Once we consider that in real life teams are not so perfectly matched, thus raising the odds of the more powerful team being successful in all four games, the fact that there are three teams with 4-0 records (Orioles, Reds, and Rangers) isn't a surprise.

The only surprise in all of this is that the Orioles (widely touted to finish last in the A.L. East) swept the Rays in a three game series in Tampa Bay and then went on to beat the Tigers at home in their fourth game of the season. But then again, the probability of a .400 team going 4-0 against a .500* team is just under 3%. Not very good odds, but something we would expect to see on occasion.

(Shout out to Tango, who raised the "on pace" problem a couple of days ago, and The Book readers who added various lucid and perceptive comments, including an XKCD cartoon. You can never go wrong using an XKCD cartoon to illustrate your point.)

Update: the New Utosky Bolshevik Show takes the Red Sox 0-3 start as its jumping off point for a post titled The Red Sox Aren't Doomed, demonstrating the same thing I did but with graphs and Python script. Score one for the NUBS.

* Changed from ".600". Comment #1 below was generated because of this typo; #2 is my detailed response.

-30-

Developing talent

2011-03-31T21:43:00.000-07:00

Today (Opening Day 2011), an excerpt from Bill James' forthcoming book Solid Fool's Gold: Detours on the Way to Conventional Wisdom appeared on the Slate site. The article is titled "Shakespeare and Verlander: Why are we so good at developing athletes and so lousy at developing writers?", and in it he provides some profound insights into discrimination in sports compared to the rest of society.

But along the way to that point, James takes a shot at the conventional wisdom that expansion dilutes the talent pool. James' contrary view is that expansion creates a short-term dilution, but over the long term more talent develops to fill the increased demand.

The thesis is built on James' assertion that raw talent is abundant, and simply needs the right opportunities -- incentives -- to be developed. In James' thought experiment, an expansion of MLB from 30 teams to 300 would over the long term have no impact on the level of talent, as talent development would expand to ensure the newly available opportunities were filled.

But can we really believe this? There has been plenty of discussion elsewhere about the distribution of baseball talent (for example, Sabernomics and The Book), all of which would, at first glance, seem to run contrary to Bill James' argument. But those talent curves are drawn based on the current system of incentives, with enough room for 25 roster players on 30 MLB teams and roughly 9,000 players in pro ball in North America and a few more thousand around the world.

Criticisms of Bill James' essay will no doubt focus on the fact that expanding the number of MLB teams beyond 30 requires some of the non-roster players currently in the minors to move up to The Show ... they aren't good enough to play today, but in an expansion environment they would be.

This might be true in the short term, but as Bill James argues, over the long haul the change in opportunities would shift, and talent would be developed to fill the new opportunities.

Currently around the margins of professional baseball are men who have given up baseball to work as a bartender, and those who have decided to pursue excellence in another sport. Players in both these groups would demonstrate different behaviour when provided a different set of incentives. The shape of the distribution curve would not change, and the average player's performance would also be unchanged, but the absolute number of players would increase.

Tom Wilhelmsen, former bartender, now pitching for the Seattle Mariners.

The latter group (the athletically gifted stars in other sports) would provide the increased numbers of players at the top end of the distribution curve, becoming the star players on Teams #31 through #300. The bartenders of today would become the focus of rigorous development regimes. It's important to remember that not only would there be 10 times more opportunities at very level, but there would also be 10 times more teams trying to succeed, and 10 times more scouts, coaches, and others keen to see their players develop into stars. And this would be repeated around the world, ensuring that the best athletes are active in the sport that provides the greatest opportunities. Given enough time, there would be enough players developed to stock 300 teams with no decrease in overall quality of play.

There are examples of this in the past. One recent example is the growth of information technology occupations -- 40 years ago, very few individuals (both in terms of absolute numbers and as a percentage of the workforce) knew how to write a computer program. But with increased job opportunities and an expansion of training, people who might otherwise chosen other occupations and career paths now can write computer programs. This does not mean the talent pool of computer programmers has been diluted; in fact, an argument could be made that the average talent and the high-end extreme of talent has increased.

Another parallel is the availability of natural resources that lay unused until somebody found a use for it. Petroleum was known to exist for centuries, but wasn't a sought-after resource until the mid-nineteenth century when a method to distill kerosene was developed, making it a cheap alternative to whale oil. In a short period of time opportunities expanded, and as a result there was a rush to develop this previously ignored resource.

-30-

Modeling: insights from the pros

2011-02-06T19:22:00.000-08:00

It's been a busy few weeks, so I've spent Super Bowl Sunday* catching up on the various blogs that I try to follow. A couple of posts from Andrew Gelman and Aleks Jakulin caught my eye: Why can't I be more like Bill James, or, The use of default and default-like models and the two-part Model Makers' Hippocratic Oath (Part 1 and Part 2).

All these posts are worth reading in their entirety, but they all boil down to the quote from George E.P. Box: "Essentially, all models are wrong, but some are useful." Knowing (or if you're the author, admitting) the limitations of the model is the most important to understanding how useful a model might be.

*Pitchers and catchers start reporting for spring training one week today!

-30-

The risks of adjusting performance stats

2011-01-28T19:05:00.000-08:00

From XKCD. Not baseball, but it brings to mind park effect, league context, etc.

Andrew Gelman's "5 Books"

2011-01-05T22:36:00.001-08:00

Andrew Gelman is one of the most interesting (IMHO) social scientist/statisticians in The Academy. Not only does he have serious statistical chops (he co-authored Bayesian Data Analysis with Carlin, Stern, and Rubin), but he also has published a raft of papers on voting patterns. His blog Statistical Modeling, Causal Inference, and Social Science -- written with his colleague Aleks Jakulin -- offers wide ranging commentary on everything from statistical theory and philosophy, to R (the statistical software), to all manner of social statistics.
Gelman was recently approached by The Browser to suggest five books on how people vote in the U.S., but instead he provided a list of five excellent books about statistics. #1 on his list: Bill James’ Baseball Abstracts 1982-1986.

-30-

The ERA distribution curve

2010-12-22T20:45:00.000-08:00

NOTE: Tango and MGL at The Book, through the "Lemonade" thread, have critiqued the analysis below and pointed out errors in my assumptions. These errors mean that my closing conclusions are wrong -- good math, but bad statistics on my part. Through this post, you will find italicized text describing my errors.
Revised January 5, 2011

J.C. Bradbury's recent blog postings (here and here) have included histograms showing the distribution of ERA across major league pitchers for the 2009 season. For his analysis, Bradbury omitted those pitchers with fewer than 100 batters faced -- in both his blog and book Hot Stove Economics, he justifies this due to the wide variance in ERA scores, much of which will be due to the small number of "samples" for each pitcher. (As we saw in my earlier post about Bo Hart, it's possible for an average player to do very well over the short term; the inverse applies too.)

But a few comments on Bradbury's blog from readers ask about the impact that those "missing cases", who account for nearly a third (28%) of all individuals who pitched in MLB in 2009, would have on the curve.

Here's the answer:

Figure 1: MLB Pitching, 2009 -- Number of Pitchers by ERA, by Number of Batters Faced

Incorporating the <100 BFP pitchers (the black chunks of each bar) adds pitchers across the whole range, although they are skewed to the right (i.e. higher ERAs). While there is a stack on the left with very low ERAs, there's a bigger group of players with an ERA greater than 10. (The highest ERA of this group was 135.00.)

NOTE 1: ERA is a poor measure to use for this type of evaluation -- for pitchers with a low number of batters faced or innings pitched, it's easy for huge numbers to appear. That 135.00 ERA is the equivalent of 15 earned runs with only a single recorded out. These exaggerated values then lead to an upward distortion of the mean for the group. A better measure would be wOBA, or other measure that resembles a probability between 0 and 1.

The table below shows the average ERA of this group and three other groups based on the number of batters faced. What we see is that the <100 BFP pitchers have a higher ERA than those who pitched more frequently. (This difference is statistically significant.) In spite of the variation in their ERAs, this group on average are less skilled than the other three groupings of pitchers.

NOTE 2: This is where I went wrong. The math is correct, but there is bias in the sample that I ignored. We can be fairly confident that pitchers who get off to a poor start won't get many opportunities to pitch -- and therefore won't get the opportunity to regress to the mean. Pitchers who do better at the start of their season will continue to pitch, and regress to the mean. This process may take them some time, which may push them over the arbitrary line of 100 batters faced. Thus the statistical significance is an artifact of the bias.

Figure 2: MLB Pitching, 2009 -- Average ERA, by Number of Batters Faced

In a thread on The Book blog that covered this same topic, I made a similar statement (reply #8): "What I’m trying to say is that our best estimate of the “true talent” of this group is an ERA of 8.11 [in the current case, 8.72], and that estimate is quite accurate". That statement got a response from Tango (reply #9) of "That is not accurate. If you look at how those pitchers who faced fewer than 100 batters did in the season preceding or the season following, THAT will give you a much better indicator of the true talent level."

So let me clarify. The average level of skill of the pitchers who faced fewer than 100 batters in 2009, is an average ERA of 8.72. Although Tango is correct in his assertion that the poorest performers would regress upwards, by the same token the best pitchers (some of whom managed a 0.00 ERA in their short stint) would get worse. But if we were to let all 188 of them continue to pitch, we can be 95% certain that the "true" ERA of the group would end up somewhere between 6.92 and 10.52.

Even the lower bound (i.e. the lowest score we would expect with our more rigorous testing) is higher than the highest range from the other groups.

NOTE 3: My statement above would be correct, if it were not for the bias in the sample. My belief had been that this group would regress not to the MLB average, but to the average of the <100 BFP pitchers. But because of the selection bias, this does not hold true.
Here's a simple example to demonstrate how this works. Think of the probability professor's favourite tool, the coin toss. If we have a penny and toss it repeatedly -- say, 10,000 times -- and recorded the result each time, the proportion of heads would very accurately reflect the true probability of the individual penny. And we'd need plenty of tosses to get an accurate measure of the single penny.

But what if instead of one penny we had 188 pennies, and we varied the number of tosses each penny got? Although the average number of tosses would be 50, some pennies might get only one toss, while others would get as many as 100 tosses. Some of those short sequences might come up all heads, while others would heavily favour the tails. On average, though, across the 188 pennies, we would find that the group average was a close reflection of "true average" of the group.

NOTE 4: The error in the initial assumption causes my coin flipping analogy to fall apart. If “success” is a head, then the coin that comes up heads >0.5 will keep being flipped, possibly with enough flips to no longer be part of the “low flip” group (over that arbitrary threshold). Meanwhile, a coin that runs tails more often will get pulled from the trials quickly, and end up <0.5 and with few flips. Thus, as a group, the coins with a smaller number of flips will end up looking worse than those that keep getting flipped. Selection bias causes an apparent difference, where none really exists.

And so it is with the pitchers in question. If they were like the other pitchers in MLB, we would expect that some of the <100 batters faced pitchers would have ERAs above the league average, while others would fall below. What we see, however, is that while there is a wide variation, the average is substantially higher than the other groups of pitchers.

NOTE 5: ...because of selection bias! The lesson: selection bias can crop up anywhere, even if you are not the one doing the selecting.

-30-

Agreeing with Bill James

2010-12-20T18:36:00.001-08:00

In 1988, the Bill James Abstract included "A Bill James Primer", with 15 statements expressing what he deemed to be useful knowledge. On that list was:
2. Talent in baseball is not normally distributed. It is a pyramid. For every player who is 10 percent above the average player, there are probably twenty players who are 10 percent below average.

I agree. (Others don't; for further discussion also see here.)

But what is this thing called "talent"? Talent is a combination of a high level of skill and sustained, consistent performance. Skill in baseball is measured through metrics such as ERA (earned run average) and OPS (on-base average plus slugging percentage) -- measures that turn counting stats into an efficiency or rate measure. While this type of measure is important, they fail to account for the fact that some players have lengthy careers, while other players have a very short MLB career. Teams will sign long-term contracts with aging superstars because the player's skill is still above average, even though they may have diminished with age.

In short, career length becomes a valid proxy for talent.

The charts below plot the number of pitchers over the period 1996-2009, by both the number of games played (which favours the relief pitchers) and innings pitched (which favours the starters). During this period a total of 2,134 individuals pitched in MLB -- but the chart shows that very few of them stuck around for any length of time.

At the head of the "games" list at 898 is the still-active Mariano Rivera, while the pitcher with the most innings over this period was Greg Maddux (2887.67 innings; and Maddux threw more than 2,100 innings before 1996, as well). These two individuals, and other Hall of Fame calibre pitchers, are out at the far right of the long tail. Close to the origin at the left are pitchers whose entire career lasted but 1/3 of an inning -- a single out.
Figure 1: Number of Pitchers, by Career Innings Pitched (1996-2009)

Figure 2: Number of Pitchers, by Career Games (1996-2009)

But what of the average skill level of those pitchers? Pitchers who get a small amount of MLB experience (fewer than 27 innings) have a higher ERA than those who get more opportunities to pitch. This group -- 27% of all MLB pitchers -- recorded an average ERA of 8.08, compared to 5.15 for the 42% who pitched between 27 to 269 innings, and 4.45 for the 27% who threw between 270 and 1349 innings. The elite, those who pitched 1350 innings and above, recorded the lowest ERA of all, 4.17.

In spite of the wide variance in the ERAs of the coffee drinkers, the differences in the mean scores are statistically significant.

Figure 3: MLB Pitchers, average ERA, by number of innings pitched (1996-2009)

In summary: there is an abundance of players who are less talented than the major league average, while at the same time the number of above-average talents is low. The distribution, at the major league level, is not normal. Just like Bill James said 22 years ago.

-30-

Angell turns comic

2010-12-16T08:34:00.000-08:00

Roger Angell, writing on the New Yorker site, cracks me up with his article "Stats".

Yes, Cliff Lee had a high UPUBB (Unexpectedly Passing Up Big Bucks), but where does it rank in the history of the game? Greg Maddux did the same in 1993, when the Yankees made him a better (well, more financially lucrative) offer than the Braves, but has anyone done a similar analysis of non-monetary influences on player signing decisions?

-30-

Slugging regression II

2010-12-10T16:33:00.000-08:00

Building on my previous post, this time around we'll look at a bigger group of hitters, those with at least 75 at-bats in both 2007 and 2008. This is a total of 360 players.

Theoretically with fewer at-bats, we would see a greater number of very high SLG values and also a larger number of below-average SLG values. But we've already seen hints that player talent gets evaluated early on (in the previous post, I identified the fact that the worst SLG in the 400+ group wasn't as awful to the same degree as the best hitters are good).

How to read the charts below: in both cases, there are 25 players plotted. Those that fall between the 100% and zero lines are regressing to the league mean. And the closer they are to the line, the bigger the regression. As shown in Figure 1, 22 of the top sluggers regressed toward the mean in 2008, 3 improved (led by Albert Pujols) and none fell below the league average.

For these players, 66% of their 2008 SLG score was accounted for by their 2007 SLG (and therefore the league average accounted for 44%).

An interesting observation is that these players are by and large the same as the 400+ AB group I dealt with in the previous post. Of the 25, 19 had 400+ ABs in both years. And of the remaining 6, 4 of the players had below 400 in 2007 and then over 400 in 2008. This group includes familiar names -- Josh Hamilton, David Murphy, and Cody Ross. All of them are young sluggers who did well in a short stint in 2007, and were given the opportunity to continue to play in 2008.

Figure 1: Top 25 SLG (2007), minimum 75 at-bats

For hitters at the bottom of the slugging table, we see a similar pattern of regression. Figure 2 shows SLG "improvement" in the opposite direction: the closer the bar gets to the bottom of the chart, the bigger the improvement. Thus of the 25 players, 17 regressed toward the mean without achieving it, and 3 others exceeded the league average (the ones who fell "below zero"). The remaining 5, on the other hand, started out below average in 2007 and fared worse in 2008.
For this group, the previous year's SLG accounted for only 55% of their 2008 SLG.

And for this group of 25, they are decidedly not the same players as the least sluggerly of the 400+ AB group. Only 1 -- Jason Kendall -- appears in both lists.

In short, the "survivor bias" that keeps good players active with opportunities to hit has an inverse impact on the players at the bottom of the stack. Unless they show a huge improvement that brings them much closer to the league average, these players seem to be destined to part-time roles.

Figure 2: Bottom 25 SLG (2007), minimum 75 at-bats

This analysis is best described as "proof of concept". Certainly any conclusions drawn should be tentatively stated, and a more robust analysis over a greater number of seasons is warranted.

-30-

Slugging regression

2010-12-10T10:06:00.001-08:00

Tango issued a multi-part challenge, of which the first part is:
1. Take the top 10 in SLG in each of the last 10 years, and tell me what the overall average SLG of these 100 players was in the following year.

The point of the challenge is to demonstrate that top performing players will regress toward the mean in subsequent seasons, and that the year under consideration accounts for, as a rule of thumb, 70% of the next season's performance, and the league average (to which their performance regresses) the other 30%.

In algebraic terms, X is predicted to be 70% when
X = (SLG2 - LSLG)/(SLG1-LSLG)
Where:
SLG1 is season 1 slugging average,
SLG2 is season 2 slugging average,
LSLG is the average league slugging average (from season 1)

Leo quickly responded (comment #1 to Tango's post), with his calculation that for slugging, 73.3% was accounted for by the player's average in the first season. To my way of thinking, the challenge has been met -- job well done, Leo!

But as I started to think about it further, I began to wonder how far through the rankings this rule of thumb holds -- as we approach the league average, the player's SLG and the league SLG become one and the same number. And at the opposite end of the scale -- the non-sluggers -- do they regress upwards towards the mean?

So my first step was to simplify the challenge, and only look at two consecutive seasons, 2007 and 2008. Using only those players who had a minimum of 400 at bats each season, I pruned the list down to 129 players in both the NL and AL. Simplifying matters further is the fact that the 2007 SLG for the NL was the same as the AL -- .423. So for my "top sluggers" I then looked at the top 25 across both leagues.

The result: for these 25 players, on average, 66% of their 2008 SLG was accounted for through their 2007 score. A few percentage points from Tango's rule of thumb, but close enough.

Charting the results shows that all but two of the top 25 sluggers regressed downwards towards the mean. And of the two, only one improved dramatically: Albert Pujols (who inched up still further in 2009, before regressing ever-so-slightly in 2010). Were Pujols not in the mix, the 2007 SLG would account for only 62% of the 2008 scores.

Another interesting observation is that of these top performers, not one fell so far in 2008 to end up with a SLG below the league average. That's not to say that it wouldn't happen, but it suggests that at the extreme end of the performance curve, as determined over the course of a full season, top performers really are above average. (NOTE: further testing required!)

But what of the other end of the ranking? I looked at the lowest performing players that I had selected, and the rule of thumb does not work. From the bottom up, the percentage explained was 87%, 84%, -4.4%, -28%, ...

At this point, I started to wonder -- why minus values? A quick check of the numbers, and I saw that these players regressed up, and to a point above the league average.

So what's different about the bottom of the range? It's simple: survivorship bias. My "sample" of 139 players who had 400+ ABs in each of 2007 and 2008, while ensuring I found the top hitters, automatically excluded those weak-slugging players who don't get many plate appearances but who collectively drag down the league average. Thus the "worst" players of the 139 with lots of ABs were not (by and large) far from the league average. The bottom of the list was Jason Kendall, who slugged .309 in 2007 for the A's and the Cubs while catching. Perform much worse than that, and you'll end up playing Triple A. Or in Kendall's case, for the Royals.

On deck: regression toward the mean, SLG with 75+ ABs.

-30-

The fundamentals

2010-12-09T09:16:00.000-08:00

Tango continues to provide the essential ingredients. First it was the run expectancy matrix, now it's bases & outs by events. These two tables are fundamentals of sabermetrics -- understanding baseball (and in particular, the costs and benefits of different strategies) starts here.

[Right: Agent Smith ponders the matrix.]

-30-

Graphing run expectancy

2010-12-07T20:49:00.000-08:00

Baseball Prospectus provided the world with the 2010 situational run expectancies, and Joshua Maciel has provided a graphic display. The graph was first posted to and then evolved as a result of feedback from readers at Tango's blog -- a fascinating process in and of itself. The graph is still a bit busy to my mind (but it's certainly not a Tufte-ian duck...), but it's a fine piece of work displaying what is arguably one of the most important pieces of baseball data.

And remember, it was George Lindsey who first published these data, back in 1963.

[Right: Joshua's run expectancy chart. Click to see it full-size, and visit his blog for all the details.]

-30-

Good math, bad statistics

2010-11-29T07:08:00.000-08:00

In the past few days, a pair of posts on other blogs caught my attention -- they seem to be coming at the same issue from different directions.

First, William R. Briggs posted "Statistics Is Not Math" (November 16, 2010). Then, Tango over at The Book posted "Detrending: when statisticians attack!" (November 24, 2010). I responded to the Tango post (comment #4), but I would like to here elaborate further.

One of the things that jumped out at me from Briggs' post was the statement that "Statistics rightly belongs to epistemology, the philosophy of how we know what we know. Probability and statistics can even be called quantitative epistemology." In other words, statistics is useful only if we have some understanding of the subject matter at hand. No amount of fancy math will help our understanding if we do not start our research with some knowledge of the topic.

In the "Detrending" post, Tango links to an unpublished (in the academic sense that it's not been published in a peer-reviewed journal) paper, by three physicists, Alexander M. Petersen , Orion Penner, and H. Eugene Stanley, entitled "Detrending career statistics in professional baseball:

Accounting for the steroids era and beyond". I may offer a longer critique of this paper at a later date, but the first thing that jumps out is an apparent ignorance of The Literature (i.e. what's been written earlier about the topic -- baseball -- from a statistical basis). This leads the authors to make conclusions that have been supported elsewhere (for example, pitcher wins are not a good measure of pitcher performance, or that standardizing allows for inter-season comparisons).

There's lots of fancy maths (some of which isn't as fancy or new-fangled as the authors seem to think) and plenty of Greek letters, but in the end it doesn't add a great deal to our understanding of baseball.

This article serves as a reminder that when we are assessing the quality of any sabermetric writing, we need to consider two factors:

1. Is the author using the appropriate statistical tools and interpreting the mathematical results correctly?

2. Does the author understand the game, including how baseball has evolved and the analytic literature that has been written over the past 50 years?

-30-

Bo knows probability

2010-11-22T21:16:00.000-08:00

Cardinals' second baseman Bo Hart

Over on 3-D Baseball, Kincaid has a nice explanation of regression to the mean in a post titled "On Correlation, Regression, and Bo Hart". The blog entry starts with the story of Bo Hart, who got called up to the Cardinals in June 2003, and promptly hit .412 over his first 75 at-bats. Since Kincaid wrote a regression to the mean article, you can guess where Hart's season went -- he finished with 286 at-bats and a .277 average.

But Kincaid flirts with a few notions that I think are worth following in a bit more detail.

First up, what are the odds that a .277 hitter will break .400 across a string of 75 at-bats?

The answer is roughly 1 in 200.

This is calculated through the fact that the binomial distribution approximates the normal distribution -- in English, if you repeat a set of binomial trials, the histogram of the count of success rates for the trials will look like the normal curve. This leads us to the probability density function, which allows us to state the probability that a value (in this case, a batting average of .412) falls at a certain point given the mean value (.277).

Using Bo Hart's season batting average of .277 as his "true talent" (or "population mean") across 75 at-bats, we can calculate the standard deviation of the distribution (0.052). We then determine that .412 lies at 2.60 standard deviations from the mean (2.60=[.412-.277]/.052). As a probability, 2.60 standard deviations is 0.5% -- or 1 in 200.

What was unusual about Bo Hart is that his 1 in 200 string of successful at-bats occurred at the beginning of his Major League career. Calculating that probability is a task for another day.

In my next post I will explore Kincaid's statements about evaluating "true talent" based on a number of observations. Specifically, I'll delve into the following questions: "At what point can we be relatively certain about our inferences of true talent based on observed performance? 75 PAs is not enough, and one million is plenty, but what about 1000?"

-30-

Sealing the exits?

2010-11-14T17:22:00.001-08:00

The Victoria Seals of the Golden Baseball League announced on Wednesday (November 10, 2010) that they were "ceasing operations". While acknowledging that the league has some serious challenges, the club was pointing more fingers at the City of Victoria, who owns and operates the Seals' home, Royal Athletic Park (RAP).

Like practically every other pro sports franchise that plays in a publicly owned facility, the Seals wanted a better deal. In the news release and press conference, the Seals stated they had asked for a larger share of the gate and concession revenue, and solutions to a variety of issues relating to the playing field, including the ability to leave the outfield fence up all summer. The City's position is that they are unwilling to have taxpayers subsidize the club, and since RAP is a multi-purpose facility available for many users their hands are tied on the field issues.

I don't know enough about the details on either side to offer an informed opinion. But I can say both sides seem to have entrenched positions, based on their specific operating requirements (for the City, that includes the political reality as well as the business side) that seem reasonable enough. In short, there may not be a middle ground that is satisfactory to both parties.

The local press has played up the distance between the Seals and the City (the news article is here, but the local daily also weighed in with this opinion piece and this more resigned article). But in so doing, the press has missed a key element that the Seals acknowledge: the Golden Baseball League is itself in a shambles. The team's press release describes the league as being in an "unstable state", but I suspect that understates the troubles.

To my way of thinking, the biggest problem is the distribution of the teams in the league. To expand the league off the mainland of North America to Victoria (on Vancouver Island) was one thing -- it guarantees a higher-per-mile travel cost (those ferries aren't cheap) and perhaps an extra hotel night. But what was the league thinking, given the evidence that the league is in a tenuous state to start with, adding clubs in Mexico (Tijuana -- not far from the mothballed San Diego club but across an international border) and in particular Hawaii (Maui)?

I have to wonder if, with their "cease operations" announcement, the management of the Seals is trying to press both the City of Victoria and the management of the Golden Baseball League. Perhaps it is just wishful thinking on my part, but if the Seals can get a more satisfactory arrangement with the City of Victoria (or another municipality in the greater Victoria area -- we have 13, after all), while pressuring the league to make some sensible choices about where the franchises are located, then perhaps we haven't seen the end of this round of professional baseball in Victoria.

Update (2010-11-15): The Globe & Mail also chimes in.

The Sabermetric bookshelf, #1

2010-11-14T17:18:00.000-08:00

The Numbers Game: Baseball's Lifelong Fascination with Statistics, by Alan Schwarz. 2004, St. Martin's Press.

In The Numbers Game, Alan Schwarz presents a well-written and tidy history of the development and evolution of the statistics that record the history of the game. Or more accurately, it's a history of baseball, and its evolution over the past century and a half, from the perspective of the numerical record and analysis of the game.

Thus Schwarz begins in the mid-nineteenth century, with Henry Chadwick's influence on the information that got recorded. But more importantly, Schwarz points out (and this becomes a recurring theme) that how the game was played was an influence on what got recorded. In the early days, the ball was "pitched" to the batter in a way that facilitated batting it -- and because pitching was secondary to hitting and fielding, there was no record of pitching performance. And as the game evolved, so did the numbers that recorded the game and got used to evaluate the players.

A second recurring theme is the weaving of the technical aspects of the statistics with the personal characters of those who developed and promoted various measures. This is very much a character-driven book -- we hear not only about the "why" of the statistics that were recorded, but the people who developed them and the means of recording them. So we hear about Al Munro Elias, Allan Roth's career with the Dodgers, Hal Richman's development of Strat-O-Matic, and George Lindsey's articles that appeared in academic journals beginning in the late 1950s. We also get an entire chapter devoted to the publication in 1969 of The Baseball Encyclopedia, and another to Bill James.

One of the things that jumps out to me is the impact that computers-- particularly the personal computer -- has had on the volume of statistics available, and the precision of the analysis that is now available. (And what is perhaps a topic for another day, the proliferation of analysts of varying quality.)

In The Numbers Game, Schwarz has written what may well be the single best introduction to sabermetrics. But it's not a technical manual that will tell you how to calculate any one statistic, or how another measure should be interpreted. Instead it's a lively history of major league baseball, and the numerical record and analysis that accompanies it.

Assessment: home run.

2010 in retrospect

2010-11-14T14:34:00.002-08:00

The 2010 MLB season has come to a close, and so begins a time of reflection and resolutions.

a) I started this blog, and rarely posted. I started a few posts, often in response to other blogs, but finished fewer still. I am confounded by the traffic on the blogosphere -- I thought I could respond thoughtfully and add something of value, but I find myself either repeating what gets said elsewhere, or sounding like a condescending pedant. Or both.

So I'll start off on a different tack, starting now.

b) I went to two MLB games in 2010. (There's nothing like living across an international border to the closest team, and 4,300 km from the "national" club). Two shutouts! Fangraphs has the results here and here.

c) The local pro team just announced they are closing up shop. More on that later.

World Series predictions

2010-10-27T10:23:00.000-07:00

The 2010 World Series starts this evening, and many pundits are making their predictions on who will win (a sample: The Baseball Analysts, New York Times, and Sports Illustrated.

But perhaps the best way to think about evaluating the pundits is contained in this article "Slick talkers and bad forecasters" by Dan Gardner. Gardner's article is about economic forecasting, but the point is relevant -- when it comes to predicting the outcome, a nuanced understanding of all of the influencing factors produces the best forecast. Or as Gardner puts it, "experts who gathered information from many sources, who were comfortable with complexity and uncertainty, and were more prepared to admit mistakes and adjust conclusions accordingly -- these were the experts worth listening to."

Skill and luck on the links

2010-08-12T09:46:00.000-07:00

Differentiating skill and luck has been a hot topic of late -- here's an article at Slate by Michael Agger titled "Dead Solid Lucky" looking at the topic in the context of golf.

The article draws heavily on an analysis by Robert A. Connolly and Richard J. Rendleman Jr., both from the Kenan-Flagler Business School, University of North Carolina. (Their article, from the Journal of the American Statistical Association, can be found in PDF format here.)

A tidy summary, quoted directly from Agger's article: "How big a deal is luck on the golf course? On average, tournament winners are the beneficiaries of 9.6 strokes of good luck. Tiger Woods' superior putting, you'll recall, gives him a three-stroke advantage per tournament. Good luck is potentially three times more important. When Connolly and Rendleman looked at the tournament results, they found that (with extremely few exceptions) the top 20 finishers benefitted from some degree of luck. They played better than predicted. So, in order for a golfer to win, he has to both play well and get lucky."

Sounds like real life. And baseball.

Skill, luck, and more than a little style

2010-07-27T13:11:00.000-07:00

Pictured: "Mr. May", Dave Winfield, comes through in the clutch in the 1992 World Series with, in his words, "One stinkin' little hit." The 11th-inning double drove in two runs and sealed the World Series win for the Blue Jays.

The BBC has posted an article and calculator ("Can chance make you a killer?") that is used to demonstrate the challenges in differentiating luck from skill. In this case, a simple scenario with fixed parameters is linked to a calculator that generates the range of possibilities.

While I'm not sure how this could be used in a baseball setting, it is a very good tool for demonstrating that it can be difficult -- particularly if you just look at "the numbers" in a selective way -- to make definitive statements about a player's ability. Such as, say, clutch hitting.

(Acknowledgement: The Book.)

Baseball imitates real life

2010-07-26T19:21:00.000-07:00

How understanding luck in baseball can help understanding real life, or at least your investment portfolio: "Untangling skill and luck" by Michael J. Mauboussin.

Mauboussin uses a variety of sabermetric analysis, including Jim Albert's 2004 paper “A Batting Average: Does It Represent Ability or Luck?” and Tango's True Talent Level analysis.

Probability of winning the division

2010-07-18T07:01:00.000-07:00

The Cool Standings website presents the probabilities that any Major League Baseball team will win the division or wild card. They also have this available for the NHL, NFL, and NBA.

If I am interpreting their methodology correctly, they are using a Pythagorean basis in a Monte Carlo simulation. (To read more, here's an ESPN article on the method.)

Interesting...

Behind in the count

2010-07-14T18:26:00.000-07:00

Time to get caught up with the goings-on elsewhere...

First up, Tom at Heureusement, ici, c'est le Blog! cleverly adapted the same Poisson method I used for perfect games to examine the plethora of no-hitters this season.

And no surprise, the number fits nicely with the outer range of the "expected" frequencies.

Tireless tennis

2010-06-24T18:01:00.000-07:00

After a bit of water-cooler chat today at work, I had planned to spend part of my evening working out the probability of the crazy Wimbledon match that saw John Isner win a marathon match against Nicolas Mahut, after the tie-breaker went 138 games and had Isner prevail 70-68.

I got home to find not one but two well-presented analyses that tackle the question, so my work would be redundant. Thus I present the following links for your reading pleasure:

1. Carl Bialik, "Isner Fitting Winner of Marathon Wimbledon Match", Wall Street Journal

2. Phil Birnbaum, "What were the odds of the 70-68 score at Wimbledon?", Sabermetric Research. (Birnbaum graciously acknowledges Bialik's article in an appended post-post foreward.)

The final summary: this was a one-in-a-million (give or take, depending on some of the assumptions) event.

Perfectly random?

2010-06-06T15:08:00.001-07:00

There has been much discussion about the recent run of perfect and almost-perfect games. A variety of hypotheses have been floated, including
pitching dominance (including a higher strike out ratio), improved defense, and the confluence of expansion, better player evaluation, and a drug-free world.

Perfect games are a rare event, so we run the risk of seeing a random cluster as a trend. There have now been 20 perfect games -- 18 in the "modern era" (since 1900), 14 since the expansion era began in 1961, and two so far in the 2010 season. How can we tell if this "streak" of two perfect games in a single season is simply a random fluctuation?

Calculating the probability of a perfect game: allowing runners

One approach is to calculate a theoretical probability based on on-base percentage (OBP). Tango has a blog entry "Perfect Game calculation" that presents one approach. His estimate was 1 perfect game per 15,000.

Another example of this appears in Mathletics by Wayne L. Winston, who calculated a probability of 0.0000489883, or 1 game in just over 20,400. Winston noted at the time the book went to press (before the 2009 season) there had been nearly 173,000 regular season games since 1900 and each game provides 2 opportunities for a perfect game (so we have 346,000 "team games"). Winston then goes on to note that we would therefore expect there to be 16.95 perfect games over that period -- almost perfectly matching the observed total of 17 to that point in time.

A side note: after Mark Buehrle's perfect game in 2009, Sky Andrecheck took a similar approach for individual players. He worked out the individual chances for the 16 modern-era players who had tossed a perfect game, based on the sum of the on-base percentage and reached-on-error percentage they allowed over their careers.

Calculating the probability of a perfect game: observed rate

A second approach to calculating the probability is to compare the observed number of perfect games and to the number of opportunities. I decided to use 1961 as year one. This was a natural point to begin -- this was the first year of baseball's expansion, and it falls mid-way between Don Larson's 1956 World Series perfecto (which had been the first in 22 years) and Jim Bunning's 90 pitch masterpiece in 1964. Between 1961 and 2009 inclusive, there were 12 perfect games -- and there were 201,506 regular season "team games". This gives us a probability of 0.00005955, or 1 perfect game every 16,790 team games played.

This method yields a result that is roughly the mid-point between Tango's and Winston's approaches.

What are the odds of two perfect games in one season?

While most statistical analysis makes the assumption that the distribution of the events is "normal", when we are dealing with rare discrete events the distribution does not resemble the normal distribution. The most common distribution used for this is the Poisson distribution.

At the probability of 1 in 16,790 across a season of 4,860 "team games" (the current number per season -- based on 2,430 games and therefore 4,860 perfect game opportunities) and 4,112 (the average number since 1961) that the probabilities, expected frequencies, and observed frequencies are as follows:

So over 50 seasons, we would predict that there would be between 1 and 2 seasons with 2 perfect games, and between 9 and 11 seasons with 1 perfect game.

So to answer the question posed in the title, the answer is "Yes -- two perfect games in one season is well within the expected distribution." The fact that 2010 has been the first season with 2 perfect games in the 50 years since 1961 fits perfectly with the expected distribution.

In future posts I will repeat the calculation of probabilities and frequencies, with modified probabilities (once the dust settles on the "correct" way to calculate the probabilities...)

Comments and questions are always welcome.

A closer look at payroll and performance

2010-06-04T22:19:00.001-07:00

A recent post on Hawkonomics presented a regression analysis of Major League Baseball team performance as a function of payroll. This post has generated some chatter in the sabermetric blogs (Sabermetric Research and The Book). If I may be so bold, the original post wasn't very well articulated, which has led to some critiques. Herein I aim to repeat the original analysis, and provide some elaboration that will aid interpretation.

It is clear that the calibre of the players on the team influences the number of wins. What is less clear is the relationship between team calibre and the total amount the team pays in salaries. We have all heard it said that rich teams "buy a championship" by loading up on highly paid free agents, but how true is it?

This relationship has been analyzed in the past. One such analysis can be found in the book The Wages of Wins by Berri, Schmidt, & Brook, and there are plenty of other sources around the sabermetric blogs. (One interesting visualization tool can be found on Ben Fry’s site.)

One of the most common ways to test a relationship between two variables is through a regression analysis. This is the approach taken by Stacey Brook over at Hawkonomics. (Note: Brook is one of the co-authors of The Wages of Wins.)

I have re-run the regression using the data supplied on his blog. I changed two things to make the results more readily comprehensible. First, I changed the salary figures to be represented as millions; thus the Yankee’s salary is expressed not as $206,333,389 but $206.3. More dramatically, I used each team’s current winning percentage and projected it out over 162 games – essentially a forecast of where the teams will end up at the end of the 2010 season if they continue at the pace established over the first ~50 games of the season.

NOTE: These transformations alter neither the “goodness of fit” of the model nor the statistical significance.

Let’s look in detail at the model that results.

1. The Correlation: The strength of the relationship between team salaries and wins

The correlation coefficient (often identified as the Pearson correlation coefficient, and represented as "R") is a unitless measure that simply tells us how much the two variables vary together in a linear manner. If they both move up in lockstep, the correlation coefficient will be 1 (the temperature outside and the amount of electricity used to run air conditioners); if one moves up while the other moves down in lockstep, the R value will be -1 (the temperature outside and the amount of natural gas burned keeping your house warm). If there is no relationship at all, then R will equal zero (the temperature outside and the amount of energy used to heat the gallons of hot water used by teenagers in the shower).

For MLB salaries and wins so far this season, the R value is 0.224. This is interpreted as being a weak linear relationship. In plainer language, the data do not really follow a linear pattern.

But there is another value -- R² or R-squared -- that gives us some language to work with. In this case, O.224 squared is 0.0503. From this, we can say that salaries improve our prediction of a team's winning success by 5.03% -- not a very big improvement at all.

This is easily seen in the X-Y chart below. Salary is plotted across the bottom, with the forecast wins up the side (I converted the team win percentages to a forecast season wins -- more on this later.) Each team is represented by one of the dark blue diamonds scattered about the chart. The predicted values derived from the regession model are shown in the form of the red dots joined by a nice straight line.

[click for a bigger version]

From this, it is easy to see that the model is not a very good predictor of actual wins. While there are some blue dots that fall close to the line, there are others that are well above or below the line. If there isn’t much difference between the actual and predicted values, we have a good model. That clearly is not the case here.

The model tells us little about what makes a winning team, because a lot of the difference in team success cannot be explained by salaries. In short, this model has no oomph.

(Back to the side note from earlier: the correlation coefficient will remain the same regardless of how we express our variables. We can convert the dollars to a percentage of the average for the season (thus the low-spending Pirates would be said to have a salary that is 39% of the average while the Yankees are spending 206% above), and the correlation coefficient remains at 0.224. Or we could convert the winning percentage to actual wins, with no change in the R value.

2a. The regression equation -- how much does a change in salaries influence wins?

The regression equation is expressed as
Y=constant+(beta*X)
Where Y is the predicted number of wins, and X is the salary. The constant is the point on the Y axis where X is equal to zero, also known as the Y intercept. The beta value is the amount that a change in X will generate in increase of one in the predicted Y value. The constant and the beta value are calculated in the model.

In this model, it becomes
Y=73.0+(0.087*X)

The interpretation: each extra $87,000 spent yields an increase in a single win, starting at a base of 73 wins. A team that spends an average amount on salaries ($90.6 million) will get an average number of wins (81).

When we start to think about this equation, it’s easy to see why the model isn’t very robust. There are some teams that are going to end the season with less than 73 wins if they keep on the way they have been. To end up below 73 wins, the model says the players should be paying the team!

2b. This year’s Moneyball teams

The model does give us a way to see which teams are getting the most production (i.e. wins) for every dollar spent – the gap between the team’s actual performance and the number of wins predicted in the model is the “residual”, and it ranges from a high of 29 wins above what the model predicts for Tampa Bay and 21 for San Diego, to a low of -34 for Baltimore and -23 for Houston.

3. Statistical significance

This model is NOT statistically significant.

So what? All this means is that if we were to use another group of 30 team wins-team salary pairs, we would likely get a different R value. We could improve the significance of the model with more team salary and wins data pairs.

But if the data points are still as dispersed as they are in this case, more data points might yield a “statistically significant” model that (and this is the important part…) has the same correlation coefficient – the model would still have no oomph. All we have then is a model that we can be confident tells us that team salary has a small relationship with being the number of wins earned.

Some parting thoughts

So we have arrived at the inescapable conclusion that this model does not tell us much about what influences wins, since there is little relationship between salaries and wins at this point in the 2010 season. The model is both weak and statistically insignificant.

The fact that the model is so weak runs counter to earlier research, which tended to find a stronger relationship. Is 2010 different, or is it just too early in the season to tell?

Comments and questions are always welcome.